Three-dimensional footprint measurement method, system, computer device, computer-readable storage medium and computer program product

Abstract

The application relates to a three-dimensional indentation measurement method, a system, a computer device, a computer readable storage medium and a computer program product. The method comprises the following steps: acquiring two-dimensional image data corresponding to a three-dimensional indentation, the two-dimensional image data being obtained by image acquisition of the three-dimensional indentation on a workpiece to be measured by an image sensor at a preset measurement pose; converting the two-dimensional image data into contour point cloud data through contour recognition and projection transformation; and converting the contour point cloud data into a preset workpiece measurement coordinate system according to pose information corresponding to the preset measurement pose, so as to obtain contour measurement data of the three-dimensional indentation. The method can improve the flexibility of indentation measurement.

Description

Technical Field

[0001] This application relates to the field of workpiece measurement technology, and in particular to a three-dimensional imprint measurement method, system, computer equipment, computer-readable storage medium, and computer program product. Background Technology

[0002] With the increasing demands for precision measurement and quality control in the manufacturing industry, workpiece surface imprint extraction and analysis technology has emerged. By accurately extracting the location, shape, and size of imprints, this technology can help optimize production processes, improve product reliability, and provide data support for subsequent repair or improvement.

[0003] In traditional techniques, imprint measurement typically employs a fixed-probe contact measurement method. This involves using a probe or sensor to directly contact the workpiece surface and scan along a fixed path to obtain local or continuous topographic data. Fixed-probe contact measurement offers high accuracy and stability, and is highly resistant to ambient light interference, making it widely used in precision manufacturing and quality control.

[0004] However, the fixed probe contact measurement method is only suitable for relatively open surfaces, such as planes or continuous curved surfaces. It cannot effectively scan and measure complex surfaces or multi-feature interlocking surfaces, and its flexibility is poor. Summary of the Invention

[0005] Therefore, it is necessary to provide a three-dimensional impression measurement method, system, computer equipment, computer-readable storage medium, and computer program product that can improve the flexibility of impression measurement in order to address the above-mentioned technical problems.

[0006] In a first aspect, this application provides a method for measuring three-dimensional impressions, including:

[0007] Two-dimensional image data corresponding to the three-dimensional imprint is obtained by an image sensor acquiring images of the three-dimensional imprint on the workpiece under a preset measurement pose.

[0008] Two-dimensional image data is converted into contour point cloud data through contour recognition and projection transformation;

[0009] Based on the pose information corresponding to the preset measurement pose, the contour point cloud data is converted to the preset workpiece measurement coordinate system to obtain the contour measurement data of the three-dimensional imprint.

[0010] In one embodiment, the workpiece to be measured is fixed by a fixing mechanism during image acquisition; before converting the contour point cloud data to a preset workpiece measurement coordinate system based on the pose information corresponding to the preset measurement pose to obtain the contour measurement data of the three-dimensional imprint, the method further includes:

[0011] Obtain the workpiece identification information of the workpiece to be tested;

[0012] A preset workpiece measurement coordinate system is determined based on at least one of the workpiece identification information and the fixing mechanism information of the fixing mechanism.

[0013] In one embodiment, the workpiece to be measured is fixed on a fixture; an image sensor is fixed on a three-dimensional pose control module; based on the pose information corresponding to a preset measurement pose, the contour point cloud data is converted to a preset workpiece measurement coordinate system to obtain the contour measurement data of the three-dimensional imprint, including:

[0014] Based on the pose information corresponding to the measured pose, a first position transformation parameter is generated, wherein the first position transformation parameter is used to characterize the position transformation relationship between the first coordinate system of the image sensor and the second coordinate system of the three-dimensional pose control module;

[0015] Based on the first position transformation parameters, the contour point cloud data is mapped to the second coordinate system to obtain the first intermediate point cloud data of the three-dimensional imprint;

[0016] Based on the preset second position transformation parameters between the three-dimensional pose control module and the fixture, the first intermediate point cloud data is mapped to the third coordinate system of the fixture to obtain the second intermediate point cloud data of the three-dimensional imprint.

[0017] Based on the workpiece identification information and the fixing mechanism information, the corresponding third position transformation parameters are determined. The second intermediate point cloud data is mapped to the preset workpiece measurement coordinate system according to the third position transformation parameters to obtain the contour measurement data of the three-dimensional imprint. The third position transformation parameters are used to characterize the position transformation relationship between the preset workpiece measurement coordinate system and the third coordinate system.

[0018] In one embodiment, the preset projection transformation relationship includes a preset homography matrix; converting two-dimensional image data into contour point cloud data through contour recognition and projection transformation includes:

[0019] Connected component contour extraction is performed on two-dimensional image data to obtain two-dimensional contour data of three-dimensional imprints;

[0020] Based on the preset homography matrix, the two-dimensional contour data is mapped to the contour point cloud data of the three-dimensional imprint.

[0021] In one embodiment, connected component contour extraction is performed on the two-dimensional image data to obtain the contour two-dimensional data of the three-dimensional imprint, including:

[0022] Threshold filtering and binarization are performed on the two-dimensional image data to obtain binary image data;

[0023] Connected component contours are extracted from the binary graph data to obtain the two-dimensional contour data of the three-dimensional imprint.

[0024] In one embodiment, the two-dimensional image data is converted into contour point cloud data through contour recognition and projection transformation, including:

[0025] The two-dimensional image data is segmented according to the preset region of interest, and at least a portion of the two-dimensional image data belonging to the preset region of interest is determined as the image data to be extracted;

[0026] By using contour recognition and projection transformation, the image data to be extracted is converted into contour point cloud data.

[0027] In one embodiment, the three-dimensional imprint corresponds to multiple measurement areas, and different measurement areas are image-acquired by an image sensor according to different measurement poses;

[0028] Before converting the contour point cloud data to a preset workpiece measurement coordinate system based on the pose information corresponding to the preset measurement pose to obtain the contour measurement data of the three-dimensional imprint, the method further includes:

[0029] Image acquisition steps: Control the image sensor to acquire images of the target measurement area corresponding to the three-dimensional imprint under the preset measurement pose, and obtain two-dimensional image data;

[0030] After controlling the image sensor to acquire images of the target measurement area corresponding to the three-dimensional imprint under a preset measurement pose, and obtaining two-dimensional image data, the method further includes:

[0031] In cases where image acquisition has not been completed in at least part of the measurement area, the target measurement area is redefined in the measurement area where image acquisition has not been completed.

[0032] Set the measurement pose corresponding to the target measurement area to the preset measurement pose, and return to execute the image acquisition step until the image acquisition of the entire measurement area is completed.

[0033] Secondly, this application also provides a three-dimensional imprint measurement system, including a fixture, an image sensor, a three-dimensional pose control module, and a controller, wherein:

[0034] The fixture is configured to hold the workpiece to be measured.

[0035] The three-dimensional pose control module is configured to adjust the image sensor to a preset measurement pose;

[0036] An image sensor is configured to acquire images of three-dimensional imprints on the workpiece under a preset measurement pose, and obtain two-dimensional image data corresponding to the three-dimensional imprints.

[0037] The controller is configured as follows:

[0038] Obtain the two-dimensional image data corresponding to the three-dimensional imprint;

[0039] Two-dimensional image data is converted into contour point cloud data through contour recognition and projection transformation;

[0040] Based on the pose information corresponding to the preset measurement pose, the contour point cloud data is converted to the preset workpiece measurement coordinate system to obtain the contour measurement data of the three-dimensional imprint.

[0041] Thirdly, this application also provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to perform the following steps:

[0042] Two-dimensional image data corresponding to the three-dimensional imprint is obtained by an image sensor acquiring images of the three-dimensional imprint on the workpiece under a preset measurement pose.

[0043] Two-dimensional image data is converted into contour point cloud data through contour recognition and projection transformation;

[0044] Based on the pose information corresponding to the preset measurement pose, the contour point cloud data is converted to the preset workpiece measurement coordinate system to obtain the contour measurement data of the three-dimensional imprint.

[0045] Fourthly, this application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, performs the following steps:

[0046] Two-dimensional image data corresponding to the three-dimensional imprint is obtained by an image sensor acquiring images of the three-dimensional imprint on the workpiece under a preset measurement pose.

[0047] Two-dimensional image data is converted into contour point cloud data through contour recognition and projection transformation;

[0048] Based on the pose information corresponding to the preset measurement pose, the contour point cloud data is converted to the preset workpiece measurement coordinate system to obtain the contour measurement data of the three-dimensional imprint.

[0049] Fifthly, this application also provides a computer program product, including a computer program that, when executed by a processor, performs the following steps:

[0050] Two-dimensional image data corresponding to the three-dimensional imprint is obtained by an image sensor acquiring images of the three-dimensional imprint on the workpiece under a preset measurement pose.

[0051] Two-dimensional image data is converted into contour point cloud data through contour recognition and projection transformation;

[0052] Based on the pose information corresponding to the preset measurement pose, the contour point cloud data is converted to the preset workpiece measurement coordinate system to obtain the contour measurement data of the three-dimensional imprint.

[0053] The aforementioned three-dimensional imprint measurement method, system, computer equipment, computer-readable storage medium, and computer program product first acquire two-dimensional image data corresponding to the three-dimensional imprint. This two-dimensional image data is obtained by an image sensor capturing images of the three-dimensional imprint on the workpiece under a preset measurement pose, thus achieving two-dimensional image acquisition of the three-dimensional imprint. Then, through contour recognition and projection transformation, the two-dimensional image data is converted into contour point cloud data, achieving contour recognition guided by the two-dimensional image data and effectively extracting the geometric features of the three-dimensional imprint. Furthermore, based on the pose information corresponding to the preset measurement pose, the contour point cloud data is transformed to a preset workpiece measurement coordinate system to obtain the contour measurement data of the three-dimensional imprint, achieving direct positioning of the three-dimensional imprint on the workpiece. On the one hand, compared to contact measurement, the combination of image recognition and pose adjustment can effectively overcome the limitations of physical conditions, enabling rapid measurement of multiple angles and features without complex clamping. This overcomes the technical shortcomings of difficulty in measuring complex surfaces or multi-feature interlocking areas due to physical contact limitations, improving the flexibility of imprint measurement. On the other hand, by guiding contour recognition and direct positioning of imprint locations through two-dimensional image data, three-dimensional imprint measurement can be achieved without feature matching and theoretical model matching, thereby avoiding positioning errors caused by model deviation or feature loss. This not only reduces the complexity of data processing but also improves the adaptability to complex surfaces or non-standard workpieces, effectively enhancing both the accuracy and flexibility of imprint measurement. Attached Figure Description

[0054] To more clearly illustrate the technical solutions in the embodiments of this application or related technologies, the drawings used in the description of the embodiments of this application or related technologies will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0055] Figure 1 This is a diagram illustrating the application environment of a three-dimensional imprint measurement method in one embodiment of this application.

[0056] Figure 2 This is a flowchart illustrating the steps for determining the preset workpiece measurement coordinates in one embodiment of this application;

[0057] Figure 3 This is a flowchart illustrating the contour recognition and projection transformation steps in one embodiment of this application;

[0058] Figure 4This is a structural block diagram of a three-dimensional imprint measurement system in one embodiment of this application;

[0059] Figure 5 This is an internal structural diagram of a computer device in one embodiment of this application. Detailed Implementation

[0060] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0061] With the increasing demands for precision measurement and quality control in the manufacturing industry, workpiece surface imprint extraction and analysis technology has emerged. This technology is primarily used to detect microscopic morphological features such as indentations, scratches, and wear on workpiece surfaces, and is widely applied in areas such as processing quality assessment, assembly fit analysis, and product life prediction. By accurately extracting the location, shape, and size of imprints, this technology can help optimize production processes, improve product reliability, and provide data support for subsequent repair or improvement.

[0062] In traditional techniques, imprint measurement can employ a fixed-probe contact measurement method. This method allows a fixed probe to scan specific local locations on the workpiece surface to obtain surface topography information; alternatively, it can be based on a fixed fixture, using a scanning probe with restricted degrees of freedom to scan and measure the surface topography of a large, continuous surface in a fixed direction. The fixed-probe contact measurement method offers high accuracy and stability, and strong resistance to ambient light interference, thus it is widely used in precision manufacturing and quality control. However, this method is only suitable for relatively open surfaces, such as planes or continuous curved surfaces. It cannot effectively scan and measure complex surfaces or surfaces with multiple interlocking features, resulting in limited flexibility.

[0063] Besides contact probes, ranging probes, such as optical scanning probes, can also be used. Regardless of whether a contact or ranging probe is used, the data measured typically only contains local geometric information of the workpiece surface; this data itself does not contain the workpiece's absolute position information within the entire workpiece. Therefore, it is often necessary to combine this with theoretical model matching to determine the imprint's position on the workpiece or its location in the workpiece's measurement coordinate system; or to directly extract the imprint from a large-scale scanned point cloud and measure its position relative to surrounding reference faces or reference feature points.

[0064] However, the location method based on theoretical model matching is often unsuitable for featureless surfaces, and may result in mismatches or no matching at all. Furthermore, directly extracting features from a large point cloud obtained in a single scan, or using a reference facet or feature for imprint location, requires secondary feature fitting. Since the reference facet or feature itself is manufactured, it may also introduce manufacturing precision errors, leading to inaccuracies in the imprint measurement results obtained using it as a reference.

[0065] Based on this, the embodiments of this application provide a three-dimensional imprint measurement method. Compared with fixed probe contact measurement or fixed probe distance measurement, the embodiments of this application can achieve multi-angle and large-range scanning by adjusting the measurement posture. Moreover, the image acquisition method can break through the structural mindset of the workpiece under test. For workpieces with complex structures, it can achieve complete measurement of all positions and angles of the workpiece under test. It can also be compatible with workpieces of different sizes and shapes for complete scanning and can also achieve online rapid batch scanning measurement.

[0066] Compared to the method of locating imprints by matching theoretical models, the embodiments of this application can effectively overcome the technical defects of registration offset and registration failure caused by insufficient registration point cloud and model features by directly locating the position of the imprint on the workpiece.

[0067] Compared to the method of locating imprints by directly extracting features or end faces from a large point cloud obtained in a single scan as a reference, the embodiments of this application transform the overall scanned point cloud to a measurement coordinate system. This eliminates the need for additional reference fitting and extraction steps and can accurately reflect the actual position of the imprint. It also has good tolerance for positioning offsets caused by manufacturing errors in the reference end face itself.

[0068] In one exemplary embodiment, such as Figure 1 As shown, a three-dimensional imprint measurement method is provided. This embodiment illustrates the application of this method to a terminal, where the terminal can be, but is not limited to, various personal computers, laptops, smartphones, tablets, IoT devices, and portable wearable devices. IoT devices can be smart speakers, smart TVs, smart air conditioners, smart in-vehicle devices, projection devices, etc. Portable wearable devices can be smartwatches, smart bracelets, head-mounted devices, etc. Head-mounted devices can be virtual reality (VR) devices, augmented reality (AR) devices, smart glasses, etc. It is understood that this method can also be applied to a server, and can also be applied to a system including a terminal and a server, and implemented through the interaction between the terminal and the server. In this embodiment, the method includes the following steps S10-S30. Wherein:

[0069] Step S10: Obtain the two-dimensional image data corresponding to the three-dimensional imprint. The two-dimensional image data is obtained by the image sensor acquiring the image of the three-dimensional imprint on the workpiece under the preset measurement pose.

[0070] The workpiece to be measured refers to the workpiece for which three-dimensional imprint measurement is required. The surface of the workpiece with the three-dimensional imprint can be a free-form surface or a combination of other three-dimensional scanning spatial structures. The combination of three-dimensional scanning spatial structures can refer to composite structures of various basic geometric shapes, such as the splicing of basic curved surfaces such as planes, cylinders, spheres, and cones; the combination of open surfaces and closed solids; and the nesting of shells with surface textures or refined structures.

[0071] A three-dimensional imprint refers to a three-dimensional mark formed on the surface of a workpiece during processing, assembly, or use. The measurement of a three-dimensional imprint primarily aims to quantify its geometric characteristics. These characteristics may include at least one of the following: depth, the position of the highest point of the contour, the position of the lowest point of the contour, and the contour dimensions. It is understood that the number of three-dimensional imprints on the workpiece can be one or more. One three-dimensional imprint may correspond to multiple two-dimensional image data, and multiple three-dimensional imprints may correspond to one two-dimensional image data; the specific method can be determined based on the actual situation, and this embodiment does not impose any limitations on this. When a three-dimensional imprint is distributed across multiple two-dimensional image data, the contour measurement data of the three-dimensional imprint can be determined based on the contour measurement data of each two-dimensional image data, and then integrated through data comparison. This embodiment does not impose any limitations on this method, but for each two-dimensional image data, the three-dimensional imprint measurement method provided in this embodiment can be used to determine the contour measurement data of its corresponding three-dimensional imprint.

[0072] An image sensor can refer to a device that converts optical images into electrical signals, such as industrial cameras and depth cameras. Two-dimensional image data can be grayscale or color images.

[0073] Measurement pose can refer to the pre-set spatial position and orientation of the image sensor. For example, the position of the camera in the workpiece measurement coordinate system (x1, y1, z1) of the workpiece can be controlled, and it can be tilted to an angle of θ° with the target surface of the workpiece, and image acquisition can be performed in this pose.

[0074] Three-dimensional imprint measurement can include two processes: data acquisition and data analysis. Data acquisition refers to the process of acquiring two-dimensional image data corresponding to the three-dimensional imprint using an image sensor. Data analysis refers to the process of analyzing the acquired two-dimensional image data of the three-dimensional imprint to determine the positional information of the three-dimensional imprint on the workpiece under test, as well as other information about the imprint. It is understood that data acquisition and data analysis can be performed continuously or discontinuously.

[0075] During data acquisition, the image sensor can be fixed to a 3D pose control module, which then regulates the sensor's pose. The 3D pose control module refers to a motion mechanism capable of precisely controlling the spatial position and orientation of the image sensor, such as a robotic arm, robot, or displacement platform with a rotating mechanism. The 3D pose control module, combined with the image sensor's workpiece scanning method, enables multi-angle, large-area scanning. It can achieve complete measurement of even complex workpieces, and is compatible with workpieces of different sizes and shapes. Furthermore, it allows for rapid online batch scanning and measurement.

[0076] In some feasible implementations, this embodiment can be applied to a data analysis terminal. It is understood that the data analysis terminal can also send control commands to devices or apparatuses involved in the data acquisition process, such as image sensors, to control the data acquisition process.

[0077] In some feasible implementations, the image sensor can be communicatively connected to a data analysis terminal. After acquiring two-dimensional image data, the image sensor can immediately send the acquired two-dimensional image data to the data analysis terminal for data analysis.

[0078] In other feasible implementations, the image sensor and the data analysis terminal can be communicatively connected to the memory. After acquiring two-dimensional image data, the image sensor can send the acquired two-dimensional image data to the memory for storage. The data analysis terminal can retrieve the two-dimensional image data from the memory for data analysis as needed.

[0079] In some feasible implementations, the workpiece under test may have multiple three-dimensional imprints. No matter how the measurement pose is adjusted, it is impossible to capture the feature information of all three-dimensional imprints in only one measurement pose. Alternatively, although there may only be one three-dimensional imprint on the workpiece under test, this imprint may span multiple surfaces of the workpiece, making it impossible to capture all the feature information of the three-dimensional imprint by adjusting the measurement pose of the image sensor.

[0080] In this situation, the data collection process can be completed first, followed by the data analysis process. Alternatively, data collection and data analysis can be performed simultaneously.

[0081] In some feasible implementations, multiple image sensors can be configured to acquire images from multiple angles. Correspondingly, multiple three-dimensional pose control modules can also be configured.

[0082] As an example, multiple measurement regions can be pre-divided from the surface of the workpiece to be measured. The feature information of the 3D imprint contained in each measurement region can be captured in one measurement pose, while the feature information of the 3D imprint contained in different measurement regions needs to be captured in different measurement poses. For example, assuming a cube-shaped workpiece to be measured, the left and right sides respectively contain the 3D imprints to be measured. The left and right sides can be regarded as different measurement regions, and the corresponding measurement poses can be set for the left side and the corresponding measurement poses can be set for the right side.

[0083] Furthermore, the image sensor can first select any measurement area that has not yet completed image acquisition, control the image sensor to adjust to the measurement pose corresponding to the measurement area, and perform image acquisition in the measurement pose; after the image sensor completes the image acquisition of the area to be measured, the data analysis terminal can obtain the two-dimensional image data corresponding to the area to be measured and perform data analysis. Regardless of whether the data analysis has started or been completed, after the image sensor completes the image acquisition of the area to be measured, the image sensor reselects the next measurement area, adjusts the pose to the measurement pose corresponding to the next measurement area, and performs image acquisition of the next measurement area.

[0084] As an example, two-dimensional image data of a three-dimensional imprint can be obtained from an image sensor by communicating with the image sensor.

[0085] As another example, two-dimensional image data of a three-dimensional imprint acquired by an image sensor and written into memory can be obtained from memory by communicating with the memory.

[0086] Step S20: Convert the two-dimensional image data into contour point cloud data through contour recognition and projection transformation.

[0087] Contour recognition can refer to the process of identifying the edge position or geometric shape of a three-dimensional imprint from an image. Contour recognition can be achieved through contour recognition algorithms, which may include edge detection, threshold segmentation, etc., and are similar to related technologies. This embodiment will not elaborate further on these technologies.

[0088] Projection transformation refers to the mathematical transformation process of mapping pixel coordinates in a two-dimensional image to three-dimensional space. The projection transformation relationship between pixel coordinates in the two-dimensional image and spatial coordinates in the three-dimensional space can be determined in advance through calibration; this embodiment does not impose any limitations on this.

[0089] In some feasible implementations, the projection transformation relationship can be a homography matrix, used to characterize the coordinate transformation relationship between pixel coordinates and point cloud coordinates in the two-dimensional image data acquired by the image sensor. The homography matrix can be determined based on camera intrinsic parameters, including focal length, distortion coefficients, etc., or it can be calibrated and determined by the user; this embodiment does not impose any restrictions on this.

[0090] As an example, after acquiring two-dimensional image data, contour recognition algorithms can be used to extract the contour pixels of the three-dimensional imprint from the two-dimensional image data; then, according to a preset projection transformation relationship, these contour pixels are projected into three-dimensional space to obtain contour point cloud data.

[0091] As another example, after obtaining the two-dimensional image data, all pixels in the two-dimensional image data can be projected into three-dimensional space according to the preset projection transformation relationship to obtain image point cloud data; then, the contour point cloud data of the three-dimensional imprint can be extracted from the image point cloud data through the contour recognition algorithm.

[0092] Step S30: Based on the pose information corresponding to the preset measurement pose, the contour point cloud data is converted to the preset workpiece measurement coordinate system to obtain the contour measurement data of the three-dimensional imprint.

[0093] Pose information can refer to a set of parameters that describe the spatial position and orientation of an image sensor.

[0094] The workpiece measurement coordinate system can refer to a predefined three-dimensional reference coordinate system fixed to the workpiece under test. This coordinate system can be established based on the fixed workpiece after its position is determined. For example, a right-handed coordinate system can be established with a reference corner point of the workpiece as the origin. It is understandable that when the image sensor is not in pose adjustment, it is in its initial pose. Under this initial pose, there is a fixed transformation relationship between the image sensor's coordinate system and the workpiece measurement coordinate system. However, as the image sensor's pose is adjusted, its coordinate system changes relative to the initial pose, causing the original fixed transformation relationship to become inapplicable. The coordinate mapping relationship must be re-established using updated pose parameters.

[0095] For example, based on the pose information corresponding to a preset measurement pose, the contour point cloud data can be transformed from the coordinate system of the image sensor to a unified workpiece measurement coordinate system to obtain the contour measurement data of the three-dimensional imprint. For instance, a corresponding rigid body transformation matrix can be determined based on the pose information, wherein the rigid body transformation matrix includes a rotation matrix and a translation vector; the contour point cloud data is rotated and translated using this rigid body transformation matrix to align it with the design datum of the workpiece, ultimately obtaining the accurate contour measurement data of the three-dimensional imprint in the workpiece measurement coordinate system.

[0096] In some feasible implementations, after converting the contour point cloud data to a preset workpiece measurement coordinate system based on the pose information corresponding to the preset measurement pose to obtain the contour measurement data of the three-dimensional imprint, the method further includes:

[0097] Based on the contour measurement data, a contour measurement result is generated, which includes at least one of the contour highest point information, contour lowest point information, and contour dimension information.

[0098] In the aforementioned three-dimensional imprint measurement method, firstly, two-dimensional image data corresponding to the three-dimensional imprint is acquired. This two-dimensional image data is obtained by an image sensor capturing images of the three-dimensional imprint on the workpiece under a preset measurement pose, thus achieving two-dimensional image acquisition of the three-dimensional imprint. Then, through contour recognition and projection transformation, the two-dimensional image data is converted into contour point cloud data, achieving contour recognition guided by the two-dimensional image data and effectively extracting the geometric features of the three-dimensional imprint. Finally, based on the pose information corresponding to the preset measurement pose, the contour point cloud data is transformed into a preset workpiece measurement coordinate system to obtain the contour measurement data of the three-dimensional imprint, achieving direct positioning of the three-dimensional imprint on the workpiece. On the one hand, compared to contact measurement, the combination of image recognition and pose adjustment can effectively overcome the limitations of physical conditions, enabling rapid measurement of multiple angles and features without complex clamping. This overcomes the technical shortcomings of difficulty in measuring complex surfaces or multi-feature interlocking areas due to physical contact limitations, improving the flexibility of imprint measurement. On the other hand, by guiding contour recognition and direct positioning of imprint locations through two-dimensional image data, three-dimensional imprint measurement can be achieved without feature matching and theoretical model matching, thereby avoiding positioning errors caused by model deviation or feature loss. This not only reduces the complexity of data processing but also improves the adaptability to complex surfaces or non-standard workpieces, effectively enhancing both the accuracy and flexibility of imprint measurement.

[0099] In one exemplary embodiment, such as Figure 2 As shown, the workpiece to be measured is fixed by a fixing mechanism during image acquisition; before converting the contour point cloud data to the preset workpiece measurement coordinate system according to the pose information corresponding to the preset measurement pose to obtain the contour measurement data of the three-dimensional imprint, the three-dimensional imprint measurement method further includes steps 202 to 204. Wherein:

[0100] Step 202: Obtain the workpiece identification information of the workpiece to be tested.

[0101] It should be noted that for each workpiece to be measured, the corresponding workpiece measurement coordinate system is usually established based on the fixed position of the workpiece. However, establishing a coordinate system for each workpiece is very time-consuming, resulting in very low efficiency in 3D imprint measurement.

[0102] In this context, a fixing mechanism refers to a device used to constrain the degrees of freedom of the workpiece under test during inspection, processing, or assembly, such as a clamp, locating pin, or vacuum suction hole. During image acquisition, to ensure the workpiece maintains a stable spatial position and orientation during measurement, a fixing mechanism is typically used to fix the workpiece to a displacement platform or fixture. For multiple workpieces with identical dimensions, shapes, and structures, their positions should also be identical after being fixed using the same fixing mechanism; therefore, the established workpiece measurement coordinate system can also be the same. Thus, for workpieces with identical dimensions, shapes, and structures, the same fixing mechanism can be used, allowing for a single coordinate system establishment. Reusing the workpiece measurement coordinate system effectively reduces the number of coordinate system establishments and improves the efficiency of 3D imprint measurement. By matching corresponding fixing mechanisms to different workpieces, the coordinate system can be established in advance before 3D imprint measurement, and the corresponding workpiece measurement coordinate system can be directly called during the 3D imprint measurement process, further improving the efficiency of 3D imprint measurement.

[0103] In some feasible implementations, workpieces that are identical in size, shape and structure can be defined as workpieces of the same model.

[0104] Workpiece identification information can refer to codes or feature data used to uniquely identify workpiece models, such as model codes, QR codes, serial numbers, etc.

[0105] For example, when fixing the workpiece to be measured, the workpiece identification information can be determined, and the determined workpiece identification information can be associated and stored with the two-dimensional image data. During the three-dimensional imprint measurement process, when acquiring the two-dimensional image data, the workpiece identification information corresponding to the two-dimensional image data can be acquired simultaneously.

[0106] Step 204: Determine a preset workpiece measurement coordinate system based on at least one of the workpiece identification information and the fixing mechanism information of the fixing mechanism.

[0107] Among them, fixed mechanism information can refer to information describing the characteristics of fixed mechanisms, such as the fixed mechanism's number, coordinates, etc.

[0108] As an example, before three-dimensional imprint measurement, one or more fixing mechanisms can be used to fix different types of workpieces to be measured. After each fixing mechanism fixes each type of workpiece to be measured, a coordinate system is established for the workpiece to be measured to obtain multiple initial workpiece measurement coordinate systems, and the correspondence between the initial workpiece measurement coordinate systems and the fixing mechanisms and models is established.

[0109] In the process of three-dimensional imprint measurement, after determining the fixing mechanism used to fix the workpiece to be measured during image acquisition and determining the workpiece identification information, the correspondence between the pre-established preset workpiece measurement coordinate system and the fixing mechanism and model can be queried. The initial workpiece measurement coordinate system corresponding to the combination of the workpiece identification information and the fixing mechanism information can be determined, and the initial workpiece measurement coordinate system corresponding to the combination of the workpiece identification information and the fixing mechanism information can be determined as the preset workpiece measurement coordinate system.

[0110] As another example, before three-dimensional imprint measurement, one or more fixing mechanisms can be used to fix the standard workpiece. After each fixing mechanism fixes the standard workpiece, a coordinate system is established for the standard workpiece to obtain multiple initial workpiece measurement coordinate systems, and the correspondence between the initial workpiece measurement coordinate systems and the fixing mechanisms is established.

[0111] In the process of 3D imprint measurement, after determining the fixing mechanism used to fix the workpiece to be measured during image acquisition, the correspondence between the pre-established initial workpiece measurement coordinate system and the fixing mechanism can be queried to determine the initial workpiece measurement coordinate system corresponding to the fixing mechanism information. After determining the workpiece identification information, the target workpiece information corresponding to the workpiece identification information can be queried. The target workpiece information may include at least one of the workpiece size information, workpiece shape information, and workpiece structure information of the workpiece to be measured. Then, based on the difference between the target workpiece information and the standard workpiece information of the standard workpiece, the initial workpiece measurement coordinate system corresponding to the fixing mechanism information is converted into a workpiece measurement coordinate system that matches the workpiece to be measured. The converted workpiece measurement coordinate system is determined as the preset workpiece measurement coordinate system.

[0112] In some feasible implementations, the fixed position and fixing mechanism of each type of workpiece on the fixture are predetermined. In this case, only the workpiece identification information is needed to determine the preset workpiece measurement coordinate system.

[0113] In this embodiment, by reusing the workpiece measurement coordinate system, the number of system establishments can be effectively reduced, thereby improving the efficiency of three-dimensional imprint measurement.

[0114] In an exemplary embodiment, the workpiece to be measured is fixed on a fixture; an image sensor is fixed on a three-dimensional pose control module; based on the pose information corresponding to a preset measurement pose, the contour point cloud data is converted to a preset workpiece measurement coordinate system to obtain the contour measurement data of the three-dimensional imprint, including steps S31 to S34. Wherein:

[0115] Step S31: Generate a first position transformation parameter based on the pose information corresponding to the measured pose. The first position transformation parameter is used to characterize the position transformation relationship between the first coordinate system of the image sensor and the second coordinate system of the three-dimensional pose control module.

[0116] It should be noted that contour point cloud data typically only contains local geometric information of the surface of the workpiece under test; this data itself does not contain its absolute position information on the entire workpiece. Currently, it is often necessary to combine this with theoretical model matching to determine the position of the imprint on the workpiece or in the workpiece measurement coordinate system; or to directly extract the imprint from a large-scale point cloud obtained from a scan and measure its position relative to surrounding reference faces or reference feature points. However, the theoretical model matching positioning method is often not suitable for featureless surfaces, and for featureless surfaces, it is easy to encounter situations where matching is not possible or incorrect. Directly extracting features from a large-scale point cloud obtained from a single scan, or using reference faces or reference features to locate the imprint position, requires secondary feature fitting. Furthermore, the reference faces or reference features themselves are machined, and may themselves have machining accuracy errors, thus leading to errors in the imprint measurement results obtained using these as references.

[0117] The first position transformation parameter can be used to characterize the spatial transformation relationship between the first coordinate system of the image sensor and the second coordinate system of the three-dimensional pose control module. In some feasible embodiments, the first position transformation parameter can be in the form of a transformation matrix, which may include at least one of a rotation matrix and a translation vector.

[0118] For example, since the image sensor is fixed on the three-dimensional pose control module, the coordinate system transformation relationship between the image sensor and the three-dimensional pose control module under the initial pose is determined. Therefore, after the pose of the image sensor changes, the rigid body transformation relationship between the first coordinate system and the second coordinate system can be calculated based on the pose information of the image sensor under the current measurement pose and the coordinate system transformation relationship between the image sensor and the three-dimensional pose control module under the preset initial pose, and the rigid body transformation relationship is determined as the first position transformation parameter.

[0119] Step S32: Based on the first position transformation parameters, the contour point cloud data is mapped to the second coordinate system to obtain the first intermediate point cloud data of the three-dimensional imprint.

[0120] For example, after obtaining the first position transformation parameters, the coordinates of each point in the contour point cloud data can be transformed based on the first position transformation parameters, and the points can be mapped from the first coordinate system of the image sensor to the second coordinate system of the three-dimensional pose control module to obtain the first intermediate point cloud data of the three-dimensional imprint in the second coordinate system.

[0121] Step S33: Based on the preset second position conversion parameters between the three-dimensional pose control module and the fixture, the first intermediate point cloud data is mapped to the third coordinate system of the fixture to obtain the second intermediate point cloud data of the three-dimensional imprint.

[0122] The second position transformation parameter can be used to characterize the spatial transformation relationship between the third coordinate system of the fixture and the second coordinate system of the three-dimensional pose control module. In some feasible embodiments, the second position transformation parameter can be in the form of a transformation matrix, which may include at least one of a rotation matrix and a translation vector.

[0123] For example, after obtaining the first intermediate point cloud data of the three-dimensional imprint in the second coordinate system, the first intermediate point cloud data can be further transformed to the third coordinate system of the fixture according to the pre-calibrated second position transformation parameters to obtain the second intermediate point cloud data of the three-dimensional imprint in the third coordinate system.

[0124] Step S34: Based on the workpiece identification information and the fixing mechanism information of the fixing mechanism, determine the corresponding third position transformation parameter, and map the second intermediate point cloud data to the preset workpiece measurement coordinate system according to the third position transformation parameter to obtain the contour measurement data of the three-dimensional imprint. The third position transformation parameter is used to characterize the position transformation relationship between the preset workpiece measurement coordinate system and the third coordinate system.

[0125] The third position transformation parameter can be used to characterize the spatial transformation relationship between the fixture's third coordinate system and the preset workpiece measurement coordinate system. In some feasible embodiments, the second position transformation parameter can be in the form of a transformation matrix, which may include at least one of a rotation matrix and a translation vector.

[0126] For example, after determining the preset workpiece measurement coordinate system based on the workpiece identification information and the fixing mechanism information, the position transformation relationship between each preset workpiece measurement coordinate system and the third coordinate system of the fixture can be calculated, and the correspondence between the workpiece identification information, the fixing mechanism information and the position transformation relationship can be established.

[0127] After obtaining the second intermediate point cloud data of the three-dimensional imprint in the third coordinate system, the corresponding relationship between the workpiece identification information, the fixed mechanism information and the position transformation relationship can be queried based on the workpiece identification information and the fixed mechanism information, and the corresponding third position transformation parameters can be determined; then the second intermediate point cloud data can be transformed from the third coordinate system of the fixture to the preset workpiece measurement coordinate system to obtain the contour measurement data of the three-dimensional imprint in the preset workpiece measurement coordinate system.

[0128] In this embodiment, through continuous coordinate transformation, the contour point cloud data can be mapped to the preset workpiece measurement coordinate system, and the whole process does not depend on the position of other feature points or end faces. Therefore, it will not be affected by the machining accuracy. Even if the workpiece itself has insufficient machining accuracy or has deviations, it will not affect the measurement accuracy of the imprint.

[0129] In one exemplary embodiment, such as Figure 3 As shown, the preset projection transformation relationship includes a preset homography matrix. Through contour recognition and projection transformation, the two-dimensional image data is converted into contour point cloud data, including steps S21 to S22. Wherein:

[0130] Step S21: Extract the contour of the connected components from the two-dimensional image data to obtain the two-dimensional contour data of the three-dimensional imprint.

[0131] Connected component contour extraction is an image processing technique used to identify interconnected pixel regions in a binary image and extract their boundary points. These interconnected pixel regions can be defined as connected components. By analyzing the adjacency relationships of pixels, such as 4-connectivity or 8-connectivity, the closed contours of the connected components can be determined.

[0132] Two-dimensional contour data can refer to the set of coordinates of the target contour describing the three-dimensional imprint in a two-dimensional image plane.

[0133] For example, a connected component analysis algorithm can be used to extract all connected component boundary points from the two-dimensional image data, and then these points can be organized into two-dimensional contour data according to a preset contour order. For instance, if three dents are detected on the surface of a workpiece, three independent sets of contour points are output, each set containing the pixel coordinates of all edges of the dent.

[0134] Step S22: Based on the preset homography matrix, the two-dimensional contour data is mapped to the contour point cloud data of the three-dimensional imprint.

[0135] The homography matrix is ​​a projection transformation matrix used to describe the mapping relationship from a two-dimensional image plane to a three-dimensional spatial plane. It can include at least one of rotation, translation, and perspective distortion parameters. The homography matrix can be obtained by calibrating the intrinsic and extrinsic parameters of the image sensor, and can also be adjusted in conjunction with the measurement pose of the image sensor.

[0136] Contour point cloud data can refer to a set of three-dimensional points obtained by spatial transformation of two-dimensional contour points, representing the geometric shape of a three-dimensional imprint in actual space.

[0137] For example, a pre-calibrated homography matrix can be used to perform a dimensionality upscaling transformation on each point in the two-dimensional contour data to obtain the contour point cloud data of the three-dimensional imprint in three-dimensional space.

[0138] In some feasible implementations, assuming any two-dimensional contour point in the two-dimensional contour data is (x,y), the two-dimensional contour point (x,y) can first be converted into homogeneous coordinates (x,y,1); then the homogeneous coordinates (x,y,1) are multiplied by the homography matrix H to obtain the three-dimensional homogeneous coordinates [X',Y',Z'] = H×[x,y,1]; assuming the Z plane is fixed, it can be further normalized to three-dimensional Euclidean coordinates (X,Y,Z) = (X' / Z', Y' / Z', 0), and the three-dimensional Euclidean coordinates are determined as contour point cloud data, which can be directly used to calculate parameters such as imprint depth and area.

[0139] In this embodiment, contour extraction is performed first, enabling contour extraction from two-dimensional data. Compared to contour extraction from point cloud data, this method is simpler and requires less computational power. Then, a projection transformation is performed on the contour data. Compared to projecting two-dimensional image data, this requires less data and lowers computational power. Therefore, this approach of extracting contours first and then performing projection transformation effectively reduces computational load, improves the efficiency of three-dimensional imprint measurement, and saves computational resources.

[0140] In an exemplary embodiment, connected component contour extraction is performed on two-dimensional image data to obtain two-dimensional contour data of a three-dimensional imprint, including steps S211 to S212. Wherein:

[0141] Step S211: Threshold filtering binarization is performed on the two-dimensional image data to obtain binary image data.

[0142] Threshold filtering binarization refers to the process of converting an image into a format that contains only black and white pixel values ​​by setting a threshold, in order to highlight target features.

[0143] Binary image data can refer to an image matrix that contains only black and white pixel values.

[0144] For example, the two-dimensional image data acquired by the image sensor can be read first, and then binarized according to a preset threshold. For instance, the pixel values ​​of pixels with brightness below a preset brightness threshold can be converted to black pixel values, and the pixel values ​​of pixels with brightness greater than or equal to the preset brightness threshold can be converted to white pixel values. As another example, the pixel values ​​of pixels with pixel values ​​below a preset pixel threshold can be converted to black pixel values, and the pixel values ​​of pixels with pixel values ​​greater than or equal to the preset brightness threshold can be converted to white pixel values.

[0145] Step S212: Extract the contour of the connected components from the binary graph data to obtain the two-dimensional contour data of the three-dimensional imprint.

[0146] For example, a connected component analysis algorithm can be used to identify all closed white or black regions from the binary graph data, then extract the boundary points of the closed white or black regions, and then organize these points into contour two-dimensional data according to a preset contour order.

[0147] In this embodiment, binarization can effectively simplify the data and improve computational efficiency, thereby improving the efficiency of three-dimensional imprint measurement.

[0148] In one exemplary embodiment, converting two-dimensional image data into contour point cloud data through contour recognition and projection transformation includes:

[0149] The two-dimensional image data is segmented according to the preset region of interest, and at least a portion of the two-dimensional image data belonging to the preset region of interest is identified as the image data to be extracted; the image data to be extracted is converted into contour point cloud data through contour recognition and projection transformation.

[0150] It should be noted that in addition to the three-dimensional imprint that needs to be measured, there may be other marks on the workpiece to be tested. During image acquisition, all these marks may be acquired into the two-dimensional image data. By matching size information, matching theoretical models, or locating feature points, the actual three-dimensional imprint that needs to be measured can be determined from multiple marks. However, this method involves a large amount of computation, low efficiency, and low cost.

[0151] The Region of Interest (ROI) refers to a predefined target detection area in an image that needs to be analyzed. By defining the ROI, spatial constraints on the two-dimensional image data can be achieved, thereby reducing computational load and improving the measurement accuracy of three-dimensional imprints. The ROI can be defined by geometric shape or semantic features, etc., and this embodiment does not impose any restrictions on this.

[0152] For example, after acquiring the two-dimensional image data, preset ROI parameters can be loaded first, and the two-dimensional image data can be masked. For example, a binary mask of the same size as the image can be created, with 1s inside the region of interest and 0s outside the region of interest. The binary mask is then bitwise ANDed with the two-dimensional image data, and the final output is the image data to be extracted, which only contains pixels of the region of interest.

[0153] Furthermore, contour recognition and projection transformation can be performed on the image data to be extracted after ROI segmentation. The sequence of three-dimensional imprint boundary points can be extracted through contour recognition, and some or all of the pixels in the image data to be extracted can be reconstructed in three dimensions through projection transformation, finally obtaining the contour point cloud data of the three-dimensional imprint contour.

[0154] In this embodiment, the spatial region for measuring the three-dimensional imprint can be constrained from the two-dimensional image data by using the region of interest. This can not only accurately locate the three-dimensional imprint to be measured and avoid false detection, but also reduce the amount of computation, improve measurement efficiency and reduce measurement cost.

[0155] In an exemplary embodiment, the three-dimensional imprint corresponds to multiple measurement areas, and different measurement areas are image-acquired by an image sensor according to different measurement poses; before converting the contour point cloud data to a preset workpiece measurement coordinate system based on the pose information corresponding to the preset measurement pose to obtain the contour measurement data of the three-dimensional imprint, the three-dimensional imprint measurement method further includes:

[0156] Image acquisition steps: Control the image sensor to acquire images of the target measurement area corresponding to the three-dimensional imprint under the preset measurement pose, and obtain two-dimensional image data.

[0157] It should be noted that the workpiece under test may have multiple 3D imprints. No matter how the measurement pose is adjusted, it is impossible to capture the feature information of all 3D imprints in only one measurement pose. Alternatively, although there may only be one 3D imprint on the workpiece under test, this 3D imprint may span multiple surfaces of the workpiece, making it impossible to capture all the feature information of the 3D imprint by adjusting the measurement pose of the image sensor.

[0158] For example, before performing data analysis on each two-dimensional image data, image acquisition of that two-dimensional image data must be completed first. During the image acquisition process, a target measurement area can be selected from the measurement areas where image acquisition is required but not yet completed. The pose information corresponding to the target measurement area is retrieved from the system database, and the pose corresponding to the pose information is determined as the preset measurement pose. The image sensor is adjusted to the preset measurement pose through the three-dimensional pose control module. Then, the image sensor is controlled to acquire images of the target measurement area corresponding to the three-dimensional imprint under the preset measurement pose, thereby obtaining the two-dimensional image data corresponding to the target measurement area.

[0159] After acquiring two-dimensional image data by controlling the image sensor to perform image acquisition on the target measurement area corresponding to the three-dimensional imprint under a preset measurement pose, the three-dimensional imprint measurement method also includes:

[0160] If image acquisition is not completed in at least part of the measurement area, the target measurement area is redefined in the measurement area where image acquisition has not been completed; the measurement pose corresponding to the target measurement area is set to the preset measurement pose, and the image acquisition step is returned to be executed until image acquisition is completed in all measurement areas.

[0161] It should be noted that for each measurement area, image acquisition must be performed first to obtain the two-dimensional image data of that measurement area before data analysis can be performed. However, for different measurement areas, image acquisition and data analysis can be performed simultaneously. For example, after the image sensor acquires the two-dimensional image data P1 of measurement area A1, the terminal can process the two-dimensional image data P1 to obtain the contour measurement data of the three-dimensional imprint in the two-dimensional image data P1. After the image sensor acquires the two-dimensional image data P1 of measurement area A1, the image sensor can also adjust its measurement pose and continue to acquire the two-dimensional image data P2 of measurement area A2. In this embodiment, the order of the image acquisition process of two-dimensional image data P2 and the data processing process of two-dimensional image data P1 is not limited.

[0162] For example, after image acquisition is completed in any measurement area, it can be detected whether there are still measurement areas where image acquisition has not been completed. If there are measurement areas where image acquisition has not been completed, one of the measurement areas where image acquisition has not been completed is selected as the target measurement area, thereby updating the target measurement area. Then, the pose information corresponding to the target measurement area is retrieved from the system database, and the pose corresponding to the pose information is determined as the preset measurement pose, thereby updating the preset measurement pose. Then, through the three-dimensional pose control module, the image sensor is adjusted to the updated preset measurement pose, and the image acquisition step is returned to perform image acquisition on the updated target measurement area. This process is repeated until image acquisition is completed in all measurement areas.

[0163] In this embodiment, by adjusting the measurement pose of the image sensor, three-dimensional imprints on various surfaces, orientations, and structures of the workpiece under test can be captured, resulting in greater measurement flexibility and the ability to capture more feature information of the three-dimensional imprints, thereby improving measurement accuracy.

[0164] In some feasible implementations, the imprint measurement method is applied to an imprint measurement system, which includes a processor, a scanning camera, a mobile robot, and a fixture. The workpiece to be measured is fixed on the fixture, and the scanning camera is mounted on the end effector of the mobile robot.

[0165] Image acquisition steps: The processor sends a movement trajectory command to the mobile robot via a connecting cable. After receiving the movement trajectory command, the mobile robot moves the measuring camera to the designated measurement position or to a specific measurement posture. After confirming that the measuring camera has reached the designated position, the processor sends a measurement trigger signal to the measuring camera via the connecting cable. After receiving the measurement trigger signal, the measuring camera captures two-dimensional image data G of the surface morphology of the workpiece to be measured within its field of view. miFor ease of understanding, let's take the example of measuring the surface of the workpiece as a freeform surface and the two-dimensional image information as a grayscale image, where i represents the sequence number of the measurement area.

[0166] After receiving the two-dimensional image data, the processor can first locate and number the regions of interest (ROIs) to be processed using pre-set ROI parameters. The two-dimensional image data G within the ROIs obtained after ROI segmentation is then processed. mi-roi It can be represented as:

[0167]

[0168] Among them, ROI i This represents the region of interest numbered i.

[0169] For two-dimensional image data G mi-roi The process involves iterating through the data and performing threshold filtering and binarization. Pixels with grayscale values ​​greater than a preset threshold are set to 1, while those with grayscale values ​​less than a preset threshold are set to 0, resulting in binary image B. mi .

[0170] Then the binary image B was analyzed. mi Perform classic connected component contour extraction, that is, extract the contours of the entire binary graph B. mi Perform a scan traversal. If the scan point itself has a value of 1, then perform an 8-way surrounding connectivity search. If there are pixels with a value of 0 in the surrounding area, then mark this scan point as a boundary point. And change and record the contour order according to the situation inside and outside the boundary where the current scan point is located, that is, the situation of the surrounding pixels with a value of 0 to the left or right or up and down of the scanned pixel.

[0171] Traversing binary graph B mi Then, the boundary points and their contour order set are obtained, and contour instances are segmented according to the contour order to obtain the edge point set of a single instance. Furthermore, the set X of edge points of the imprint instance is obtained according to the contour order. i [u i , v i ].

[0172] Using the set of edge points of imprint instances X i [u i , v i [The homography matrix V between pixels in a two-dimensional image and point cloud data is used to establish a one-to-one correspondence between pixel coordinates and point cloud coordinates.] i , will Xi[u i , v i Each point in the graph is mapped to a 3D space to obtain contour point cloud data t. i [xi, yi, zi]. Contour point cloud data t i [xi, yi, zi] can be represented as:

[0173]

[0174] After obtaining the contour point cloud data, the current pose information of the mobile robot can be acquired. This pose information may include at least one of the end-effector coordinate transformation parameter [R|t] and joint motion angles. Taking the end-effector coordinate transformation parameter [R|t] as an example, R represents the rotation matrix, and t represents the displacement vector. The contour point cloud data can be transformed to the mobile robot's coordinate system based on this. The contour point cloud data t in the mobile robot's coordinate system. i-robot It can be represented as:

[0175]

[0176] The processor checks whether scanning of all measurement areas is complete. If not, it returns to the image acquisition step until image acquisition of all measurement areas is complete; if complete, it proceeds to the next step.

[0177] The processor processes the contour point cloud data mapped to the coordinate system of the mobile robot. Based on the relatively fixed relationship between the mobile robot and the fixture, it uses a pre-set fixed transformation matrix K to uniformly map the contour point cloud data to the coordinate system of the fixture. The contour point cloud data t in the fixture's coordinate system... i-fix It can be represented as:

[0178]

[0179] Furthermore, based on the workpiece model number H, the pre-set coordinate system transformation relationship [H, E] between the fixture and the workpiece model is retrieved. H ], to obtain the fixed transformation matrix E H According to the fixed transformation matrix E H The contour point cloud data in the coordinate system of the unified mapping fixture is transferred to the preset workpiece measurement coordinate system. Contour measurement data t in the workpiece measurement coordinate system. i-measure It can be represented as:

[0180]

[0181] Integrate the contour measurement data of all three-dimensional imprints extracted from all measurement areas, analyze the spatial relationship of the imprints relative to the workpiece as a whole in the workpiece measurement coordinate system, and output a measurement report.

[0182] It should be understood that although the steps in the flowcharts of the above embodiments are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the above embodiments may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0183] Based on the same inventive concept, this application also provides a three-dimensional impression measuring device for implementing the three-dimensional impression measuring method described above. The solution provided by this device is similar to the solution described in the above method; therefore, the specific limitations of one or more embodiments of the three-dimensional impression measuring device provided below can be found in the limitations of the three-dimensional impression measuring method described above, and will not be repeated here.

[0184] In one exemplary embodiment, such as Figure 4 As shown, a three-dimensional imprint measurement system is provided, including: a fixture 402, an image sensor 404, a three-dimensional pose control module 406, and a controller 408, wherein:

[0185] The fixture 402 is configured to fix the workpiece 410 to be measured;

[0186] The three-dimensional pose control module 406 is configured to adjust the image sensor to a preset measurement pose.

[0187] Image sensor 404 is configured to acquire images of three-dimensional imprints on workpiece 410 under a preset measurement pose, and obtain two-dimensional image data corresponding to the three-dimensional imprints.

[0188] Controller 408 is configured as follows:

[0189] Obtain the two-dimensional image data corresponding to the three-dimensional imprint;

[0190] Two-dimensional image data is converted into contour point cloud data through contour recognition and projection transformation;

[0191] Based on the pose information corresponding to the preset measurement pose, the contour point cloud data is converted to the preset workpiece measurement coordinate system to obtain the contour measurement data of the three-dimensional imprint.

[0192] In one exemplary embodiment, the workpiece to be measured is fixed by a fixing mechanism during image acquisition; the controller 408 is further configured to:

[0193] Obtain the workpiece identification information of the workpiece to be tested;

[0194] A preset workpiece measurement coordinate system is determined based on at least one of the workpiece identification information and the fixing mechanism information.

[0195] In one exemplary embodiment, controller 408 is further configured to:

[0196] Based on the pose information corresponding to the measured pose, a first position transformation parameter is generated, wherein the first position transformation parameter is used to characterize the position transformation relationship between the first coordinate system of the image sensor and the second coordinate system of the three-dimensional pose control module;

[0197] Based on the first position transformation parameters, the contour point cloud data is mapped to the second coordinate system to obtain the first intermediate point cloud data of the three-dimensional imprint;

[0198] Based on the preset second position transformation parameters between the three-dimensional pose control module and the fixture, the first intermediate point cloud data is mapped to the third coordinate system of the fixture to obtain the second intermediate point cloud data of the three-dimensional imprint.

[0199] Based on the workpiece identification information and the fixing mechanism information, the corresponding third position transformation parameters are determined. The second intermediate point cloud data is mapped to the preset workpiece measurement coordinate system according to the third position transformation parameters to obtain the contour measurement data of the three-dimensional imprint. The third position transformation parameters are used to characterize the position transformation relationship between the preset workpiece measurement coordinate system and the third coordinate system.

[0200] In an exemplary embodiment, the preset projection transformation relationship includes a preset homography matrix; the controller 408 is further configured to:

[0201] Connected component contour extraction is performed on two-dimensional image data to obtain two-dimensional contour data of three-dimensional imprints;

[0202] Based on the preset homography matrix, the two-dimensional contour data is mapped to the contour point cloud data of the three-dimensional imprint.

[0203] In one exemplary embodiment, controller 408 is further configured to:

[0204] Threshold filtering and binarization are performed on the two-dimensional image data to obtain binary image data;

[0205] Connected component contours are extracted from the binary graph data to obtain the two-dimensional contour data of the three-dimensional imprint.

[0206] In one exemplary embodiment, controller 408 is further configured to:

[0207] The two-dimensional image data is segmented according to the preset region of interest, and at least a portion of the two-dimensional image data belonging to the preset region of interest is determined as the image data to be extracted;

[0208] By using contour recognition and projection transformation, the image data to be extracted is converted into contour point cloud data.

[0209] In one exemplary embodiment, the three-dimensional imprint corresponds to multiple measurement areas, and different measurement areas are image-acquired by an image sensor according to different measurement poses; the controller 408 is also configured to:

[0210] Image acquisition steps: Control the image sensor to acquire images of the target measurement area corresponding to the three-dimensional imprint under the preset measurement pose, and obtain two-dimensional image data;

[0211] In cases where image acquisition has not been completed in at least part of the measurement area, the target measurement area is redefined in the measurement area where image acquisition has not been completed.

[0212] Set the measurement pose corresponding to the target measurement area to the preset measurement pose, and return to execute the image acquisition step until the image acquisition of the entire measurement area is completed.

[0213] In one exemplary embodiment, a computer device is provided, which may be a terminal, and its internal structure diagram may be as follows: Figure 5As shown, the computer device includes a processor, memory, input / output interfaces, a communication interface, a display unit, and an input device. The processor, memory, and input / output interfaces are connected via a system bus, and the communication interface, display unit, and input device are also connected to the system bus via the input / output interfaces. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The input / output interfaces are used for exchanging information between the processor and external devices. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, mobile cellular networks, Near Field Communication (NFC), or other technologies. When the computer program is executed by the processor, it implements a data batch processing method. The display unit is used to form a visually visible image and can be a display screen, a projection device, or a virtual reality imaging device. The display screen can be an LCD screen or an e-ink screen. The input device of the computer device can be a touch layer covering the display screen, or buttons, trackballs, or touchpads set on the casing of the computer device, or external keyboards, touchpads, or mice, etc.

[0214] Those skilled in the art will understand that Figure 5 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0215] In one exemplary embodiment, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in the above-described method embodiments.

[0216] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon that, when executed by a processor, implements the steps in the above method embodiments.

[0217] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps in the above method embodiments.

[0218] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, artificial intelligence (AI) processors, etc., and are not limited to these.

[0219] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0220] The above embodiments are merely illustrative of several implementation methods of this application, and their descriptions are relatively specific and detailed. However, they should not be construed as limiting the scope of this application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A method for measuring three-dimensional impressions, characterized in that, The method includes: Two-dimensional image data corresponding to the three-dimensional imprint is obtained by an image sensor fixed on the three-dimensional pose control module under a preset measurement pose, which acquires images of the three-dimensional imprint on the workpiece to be measured, which is fixed on the fixture by a fixing mechanism. At least a portion of the two-dimensional image data belonging to the preset region of interest is identified as the image data to be extracted; The image data to be extracted is converted into contour point cloud data through contour recognition and projection transformation; Based on the pose information corresponding to the preset measurement pose, a first position transformation parameter is generated. Based on the first position transformation parameter, the contour point cloud data is mapped to the coordinate system of the three-dimensional pose control module. Then, based on the preset second position transformation parameter between the three-dimensional pose control module and the fixture, it is mapped to the fixture coordinate system. Based on the workpiece identification information of the workpiece to be measured and the fixing mechanism information of the fixing mechanism, the corresponding third position transformation parameter is matched to transform the point cloud in the fixture coordinate system to the preset workpiece measurement coordinate system, thereby obtaining the contour measurement data of the three-dimensional imprint.

2. The method according to claim 1, characterized in that, Before generating a first position transformation parameter based on the pose information corresponding to the preset measurement pose, mapping the contour point cloud data to the coordinate system of the three-dimensional pose control module based on the first position transformation parameter, mapping it to the fixture coordinate system based on the preset second position transformation parameter between the three-dimensional pose control module and the fixture, and matching the corresponding third position transformation parameter based on the workpiece identification information of the workpiece to be measured and the fixing mechanism information of the fixing mechanism to transform the point cloud in the fixture coordinate system to the preset workpiece measurement coordinate system to obtain the contour measurement data of the three-dimensional imprint, the method further includes: Obtain the workpiece identification information of the workpiece to be tested; A preset workpiece measurement coordinate system is determined based on at least one of the workpiece identification information and the fixing mechanism information of the fixing mechanism.

3. The method according to claim 2, characterized in that, The process involves generating a first position transformation parameter based on the pose information corresponding to a preset measurement pose, mapping the contour point cloud data to the coordinate system of the 3D pose control module based on the first position transformation parameter, mapping it to the fixture coordinate system based on a second position transformation parameter between the preset 3D pose control module and the fixture, and matching the corresponding third position transformation parameter based on the workpiece identification information of the workpiece to be measured and the fixing mechanism information of the fixing mechanism to transform the point cloud in the fixture coordinate system to the preset workpiece measurement coordinate system, thereby obtaining the contour measurement data of the 3D imprint, including: Based on the pose information corresponding to the measured pose, a first position transformation parameter is generated, wherein the first position transformation parameter is used to characterize the position transformation relationship between the first coordinate system of the image sensor and the second coordinate system of the three-dimensional pose control module. Based on the first position transformation parameters, the contour point cloud data is mapped to the second coordinate system to obtain the first intermediate point cloud data of the three-dimensional imprint; According to the preset second position transformation parameters between the three-dimensional pose control module and the fixture, the first intermediate point cloud data is mapped to the third coordinate system of the fixture to obtain the second intermediate point cloud data of the three-dimensional imprint. Based on the workpiece identification information and the fixing mechanism information of the fixing mechanism, the corresponding third position transformation parameter is determined. The second intermediate point cloud data is mapped to the preset workpiece measurement coordinate system according to the third position transformation parameter to obtain the contour measurement data of the three-dimensional imprint. The third position transformation parameter is used to characterize the position transformation relationship between the preset workpiece measurement coordinate system and the third coordinate system.

4. The method according to claim 1, characterized in that, The preset projection transformation relationship includes a preset homography matrix; the step of converting the image data to be extracted into contour point cloud data through contour recognition and projection transformation includes: Connected component contour extraction is performed on the image data to be extracted to obtain the two-dimensional contour data of the three-dimensional imprint; Based on a preset homography matrix, the two-dimensional contour data is mapped to the contour point cloud data of the three-dimensional imprint.

5. The method according to claim 4, characterized in that, The step of extracting the connected component contours from the image data to be extracted to obtain the two-dimensional contour data of the three-dimensional imprint includes: The image data to be extracted is subjected to threshold filtering and binarization to obtain binary image data; The connected component contours of the binary graph data are extracted to obtain the two-dimensional contour data of the three-dimensional imprint.

6. The method according to claim 1, characterized in that, The method further includes: The two-dimensional image data is segmented according to a preset region of interest.

7. The method according to any one of claims 1 to 6, characterized in that, The three-dimensional imprint corresponds to multiple measurement areas, and different measurement areas are captured by the image sensor according to different measurement poses; Before generating a first position transformation parameter based on the pose information corresponding to the preset measurement pose, mapping the contour point cloud data to the coordinate system of the three-dimensional pose control module based on the first position transformation parameter, mapping it to the fixture coordinate system based on the preset second position transformation parameter between the three-dimensional pose control module and the fixture, and matching the corresponding third position transformation parameter based on the workpiece identification information of the workpiece to be measured and the fixing mechanism information of the fixing mechanism to transform the point cloud in the fixture coordinate system to the preset workpiece measurement coordinate system to obtain the contour measurement data of the three-dimensional imprint, the method further includes: Image acquisition steps: Control the image sensor to acquire images of the target measurement area corresponding to the three-dimensional imprint under a preset measurement pose, and obtain two-dimensional image data; After controlling the image sensor to acquire images of the target measurement area corresponding to the three-dimensional imprint under a preset measurement pose to obtain two-dimensional image data, the method further includes: In cases where image acquisition has not been completed in at least part of the measurement area, the target measurement area is redefined in the measurement area where image acquisition has not been completed. Set the measurement pose corresponding to the target measurement area to the preset measurement pose, and return to execute the image acquisition step until the image acquisition of the entire measurement area is completed.

8. A three-dimensional imprint measurement system, characterized in that, The system includes a fixture, a fixing mechanism, an image sensor, a three-dimensional pose control module, and a controller, wherein: The fixing mechanism is disposed on the fixture and configured to fix the workpiece to be tested to the fixture; The three-dimensional pose control module is configured to adjust the image sensor to a preset measurement pose; The image sensor is fixed on the three-dimensional pose control module and is configured to acquire images of the three-dimensional imprint on the workpiece under the preset measurement pose, so as to obtain two-dimensional image data corresponding to the three-dimensional imprint. The controller is configured as follows: Obtain the two-dimensional image data corresponding to the three-dimensional imprint; At least a portion of the two-dimensional image data belonging to the preset region of interest is identified as the image data to be extracted; The image data to be extracted is converted into contour point cloud data through contour recognition and projection transformation; Based on the pose information corresponding to the preset measurement pose, a first position transformation parameter is generated. Based on the first position transformation parameter, the contour point cloud data is mapped to the coordinate system of the three-dimensional pose control module. Then, based on the preset second position transformation parameter between the three-dimensional pose control module and the fixture, it is mapped to the fixture coordinate system. Based on the workpiece identification information of the workpiece to be measured and the fixing mechanism information of the fixing mechanism, the corresponding third position transformation parameter is matched to transform the point cloud in the fixture coordinate system to the preset workpiece measurement coordinate system, thereby obtaining the contour measurement data of the three-dimensional imprint.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 7.

11. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 7.

Images