Full-automatic measurement method of an image measuring instrument based on a 3D CAD model of space

Abstract

The present application relates to a fully automatic measurement method for an image measuring instrument based on a three-dimensional CAD model of a space, including: importing a three-dimensional image of a part, where the three-dimensional image includes a three-dimensional CAD model of the space; taking the shooting direction determined based on the part features in the three-dimensional image as the first direction, determining the first origin of the first coordinate axis in the first coordinate system according to the part features in the three-dimensional image, and determining the origins of other coordinate axes based on the part features for determining the first origin or other part features in the three-dimensional image, and fitting each coordinate axis of the first coordinate system based on the first direction and the second direction; determining the origins of each coordinate axis of each of the first coordinate systems based on the part features in the three-dimensional image; generating a measurement program file based on the first coordinate system. Using this method can improve the processing efficiency while ensuring the measurement accuracy.

Description

Technical Field

[0001] The present application relates to the field of intelligent manufacturing equipment industry, and particularly to a full-automatic measurement method for an image measuring instrument based on a three-dimensional CAD model in space. Background Art

[0002] An image measuring instrument can achieve precise measurement of surface dimensions, profiles, angles and positions, geometric tolerances, etc. of various complex parts. Specifically, the image measuring instrument can extract geometric features (such as planes, lines and points) of parts, and calculate length or angle information, so as to be able to judge whether the machining accuracy of parts meets the design requirements. However, when batch measuring complex parts, a large number of repetitive operations are required, which is not conducive to improving the measurement efficiency.

[0003] In the traditional technology, in order to improve the measurement efficiency, before starting batch measurement, template programming can be carried out according to the design requirements of parts, a supporting measurement program can be customized, and during the subsequent measurement process, the image measuring instrument can run the measurement program to automatically measure the workpiece according to the formulated measurement steps.

[0004] However, currently, there are the following problems with using physical workpieces for template programming in image measuring instruments: The dimensional accuracy of the physical workpieces themselves may not be standard, and the images are blurred due to poor focusing when photographing the physical workpieces. These situations are likely to cause mistakes in subsequent automatic measurement. Some image measuring instruments can use CAD drawings for template programming. However, CAD drawings have only one measurement plane and cannot perform template programming for different shooting directions.

[0005] In summary, there is an urgent need for a method that can improve the processing efficiency while ensuring the measurement accuracy. Summary of the Invention

[0006] Based on this, in view of the above technical problems, it is necessary to provide a full-automatic measurement method for an image measuring instrument based on a three-dimensional CAD model in space that can improve the processing efficiency while ensuring the measurement accuracy.

[0007] In a first aspect, the present application provides a full-automatic measurement method for an image measuring instrument based on a three-dimensional CAD model in space, and the method includes:

[0008] Import a three-dimensional image of a part, where the three-dimensional image includes a three-dimensional CAD model in space;

[0009] Take the shooting direction determined based on the part features in the three-dimensional image as the first direction, determine a second direction according to the part features in the three-dimensional image, and fit the coordinate axes of the first coordinate system based on the first direction and the second direction;

[0010] Determine the first origin of the first coordinate axis in the first coordinate system based on the part features in the three-dimensional image, and determine the origins of other coordinate axes based on the part features for determining the first origin or other part features in the three-dimensional image;

[0011] Generate a measurement program file based on the first coordinate system;

[0012] Obtain the captured image to be processed, and establish a second coordinate system based on the image to be processed;

[0013] Measure the parts in the image to be processed based on the measurement program and the second coordinate system.

[0014] In one embodiment, taking the shooting direction determined based on the part features in the three-dimensional image as the first direction includes at least one of the following methods:

[0015] Determine the shooting direction based on the selected straight line in the three-dimensional image, and take the shooting direction as the first direction;

[0016] Determine the shooting direction based on the selected first plane in the three-dimensional image, and take the shooting direction as the first direction, including: determining at least 3 non-collinear points in the selected part in the three-dimensional image; generating a first plane based on the at least 3 non-collinear points, and determining the normal vector direction of the first plane as the shooting direction.

[0017] In one embodiment, determining the second direction according to the part features in the three-dimensional image includes at least one of the following methods:

[0018] Fit a first straight line based on at least two selected points in the three-dimensional image, and determine the direction of the first straight line as the second direction;

[0019] Take the selected straight line in the three-dimensional image as the first straight line, and determine the direction of the first straight line as the second direction.

[0020] In one embodiment, determining the first origin of the first coordinate axis in the first coordinate system based on the part features in the three-dimensional image, and determining the origins of other coordinate axes based on the part features for determining the first origin or other part features in the three-dimensional image includes:

[0021] Obtain the origin of the X-axis of the first coordinate system based on the projection of the first selected point in the three-dimensional image on the X-axis of the first coordinate system;

[0022] Generate the first option corresponding to the first selected point and the first option corresponding to the first straight line, and determine the origin of the Y-axis of the first coordinate system according to the first selection result of the first option corresponding to the first selected point and the first option corresponding to the first straight line. Wherein, when the first selection result is the first option corresponding to the first selected point, the origin of the Y-axis of the first coordinate system is obtained based on the projection of the first selected point on the Y-axis of the first coordinate system; when the first selection result is the first option corresponding to the first straight line, the origin of the Y-axis of the first coordinate system is obtained based on the projection of the first straight line on the Y-axis of the first coordinate system;

[0023] Generate the second option corresponding to the first selected point, the second option corresponding to the first straight line, and the option corresponding to the first plane, and determine the origin of the Z-axis of the first coordinate system according to the second selection result of the second option corresponding to the first selected point, the second option corresponding to the first straight line, and the option corresponding to the first plane. Wherein, when the second selection result is the second option corresponding to the first selected point, the origin of the Z-axis of the first coordinate system is obtained based on the projection of the first selected point on the Z-axis of the first coordinate system; when the second selection result is the second option corresponding to the first straight line, the origin of the Z-axis of the first coordinate system is obtained based on the projection of the first straight line on the Z-axis of the first coordinate system; when the second selection result is the option corresponding to the first plane, the origin of the Z-axis of the first coordinate system is obtained based on the projection of the first plane on the Z-axis of the first coordinate system.

[0024] In one embodiment, the generating the measurement program file based on the first coordinate system includes:

[0025] Determine the extraction method of the target feature in the part based on the first coordinate system and the three-dimensional image, including: performing simulation based on the first coordinate system and the three-dimensional image to obtain a simulation image when shooting along the shooting direction; determining the extraction method of the target feature in the part based on the simulation image and the three-dimensional image;

[0026] Determine the measurement method of the target feature in the part in the three-dimensional image;

[0027] Generate a measurement program file based on the extraction method of the target feature, the measurement method of the target feature, and the first coordinate system.

[0028] In one embodiment, the extraction method of the target feature includes at least one of a feature point extraction method, a feature straight line extraction method, a feature circle extraction method, and a feature plane extraction method;

[0029] The feature point extraction method includes at least one of a method of determining feature points based on manual selection and a method of determining feature points based on scanning. The method of determining feature points based on scanning includes determining two first candidate points selected in the three-dimensional image and scanning the boundary between the two first candidate points to obtain each feature point;

[0030] The feature line extraction method includes: determining at least two second candidate points selected in the three-dimensional image, and based on the at least two second candidate points, displaying a first point selection area in the simulation image, and fitting based on each feature point selected in the first point selection area to obtain a feature line;

[0031] The feature circle extraction method includes: determining a feature circle selected in the three-dimensional image, and obtaining a target feature circle based on the selected feature circle, including any of the following methods: generating a second point selection area based on the feature circle, and fitting the feature points identified in the second point selection area to obtain a feature circle, where the second point selection area is a sector point selection area or an annular point selection area; identifying a point set at the junction based on multiple selected points selected near the feature circle, and fitting based on the point set to obtain a target feature circle; and obtaining the number of set scan lines, generating each scan line passing through the feature circle in the simulation image, generating each feature point based on the scan line, and fitting based on the generated each feature point to obtain a target feature circle;

[0032] The feature plane extraction method includes: determining a plane selected in the three-dimensional image as a feature plane.

[0033] In one embodiment, the method of measuring the target feature in the part in the three-dimensional image includes:

[0034] Determining at least one of a length measurement method, an angle measurement method, and a height measurement method for the target feature in the part in the three-dimensional image;

[0035] The length measurement method is used to measure the length of the target feature;

[0036] The angle measurement method is used to measure the angle of the target feature;

[0037] The height measurement method includes measuring the height corresponding to the target feature based on a laser ranging device or measuring the height of the target feature based on a camera. Measuring the height of the target feature based on a camera includes: moving the camera above the target feature and moving the camera along the Z axis, and when the quality of the image collected by the camera meets the image quality requirement, the height of the camera is used as the height of the target feature.

[0038] In one embodiment, the generation of a measurement program based on the extraction method of the target feature, the measurement method of the target feature, and the first coordinate system includes:

[0039] In the case where the image measuring instrument can only take pictures along one shooting direction, for different measurement surfaces, a measurement program is generated based on the extraction method of the target feature, the measurement method of the target feature, and the first coordinate system;

[0040] In the case where the image measuring instrument includes multiple shooting directions, the extraction method of the target feature, the measurement method of the target feature, and the first coordinate system corresponding to each different measurement surface are stored in a measurement program.

[0041] In one embodiment, the overall contour of the part in the shooting direction is stored when the measurement program is stored; the establishment of the second coordinate system based on the image to be processed includes:

[0042] Identifying the pose of the part in the image to be processed based on the overall contour;

[0043] Generating a second coordinate system based on the recognition result of the pose and the first coordinate system corresponding to the shooting direction.

[0044] In one embodiment, the establishment of the second coordinate system based on the image to be processed includes:

[0045] Determining a fourth direction and a fifth direction according to the part features in the image to be processed, determining a sixth direction according to the shooting direction, and fitting each coordinate axis of the second coordinate system based on the fourth direction, the fifth direction, and the sixth direction;

[0046] Determining the origin of each coordinate axis of the second coordinate system based on the part features in the image to be processed;

[0047] Determining the second coordinate system based on each coordinate axis of the second coordinate system and the origin of each coordinate axis of the second coordinate system.

[0048] The fully automatic measurement of the above-mentioned image measuring instrument based on the spatial three-dimensional CAD model imports a three-dimensional image of a part, the three-dimensional image including a spatial three-dimensional CAD model; uses a shooting direction determined based on the part features in the three-dimensional image as a first direction, and determines a second direction based on the part features in the three-dimensional image, and obtains the coordinate axes of the first coordinate system based on the first direction and the second direction; determines a first origin of one of the coordinate axes of the first coordinate system based on the part features in the three-dimensional image, and determines the origins of the other coordinate axes based on the first origin or the part features in the three-dimensional image; generates a measurement program based on the first coordinate system, obtains a captured image to be processed, and establishes a second coordinate system based on the image to be processed; measures the part in the image to be processed based on the measurement program and the second coordinate system, wherein template programming is performed directly in the three-dimensional image, and template programming can be performed based on different shooting directions, thereby improving efficiency. Programming directly based on the three-dimensional image file of the part has higher accuracy, and establishing the first coordinate system based on the three-dimensional image ensures the accuracy of the first coordinate system, laying the foundation for the measurement accuracy of the subsequent measurement process. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] In order to more clearly illustrate the technical solutions in the embodiments of the present application or related technologies, the following briefly introduces the drawings required for use in the embodiments of the present application or related technical descriptions. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying any creative work.

[0050] Figure 1 FIG2 is an application environment diagram of a method for generating a measurement program of an image measuring instrument based on a spatial three-dimensional CAD model in one embodiment;

[0051] Figure 2 1 is a flow chart of a method for generating a measurement program of an image measuring instrument based on a spatial three-dimensional CAD model in one embodiment;

[0052] Figure 3 A flowchart of a first direction determination step in one embodiment;

[0053] Figure 4 1 is a flow chart of a fully automatic measurement method using an image measuring instrument based on a spatial three-dimensional CAD model in one embodiment;

[0054] Figure 5 Schematic diagram of a flow chart of a fully automatic measurement method using an image measuring instrument based on a spatial three-dimensional CAD model in another embodiment;

[0055] Figure 6It is a structural block diagram of a device for generating an image measuring instrument measurement program based on a three-dimensional CAD model in an embodiment;

[0056] Figure 7 It is a structural block diagram of a full-automatic measuring device of an image measuring instrument based on a three-dimensional CAD model in an embodiment;

[0057] Figure 8 It is an internal structure diagram of a computer device in an embodiment. Specific embodiments

[0058] In order to make the objectives, technical solutions and advantages of the present application clearer, the present application will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0059] The full-automatic measurement method of the image measuring instrument based on the three-dimensional CAD model of the present application may include: the method for generating the image measuring instrument measurement program based on the three-dimensional CAD model and the automatic measurement method of the image measuring instrument based on the three-dimensional CAD model. Before batch measurement, the method for generating the image measuring instrument measurement program based on the three-dimensional CAD model is executed, including: performing template programming on the three-dimensional image of the part, for example, the three-dimensional CAD model of the part, and defining the corresponding first coordinate system during template programming, and customizing the allocated measurement program based on the first coordinate system. Subsequently, the automatic measurement method of the image measuring instrument based on the three-dimensional CAD model is executed, that is, during the measurement process, the image measuring instrument can run the measurement program to automatically measure the part according to the planned measurement steps. In this way, directly performing template programming in the three-dimensional image can perform template programming based on different shooting directions, improving the efficiency, and directly programming based on the three-dimensional image file of the part, with higher accuracy, and establishing the first coordinate system based on the three-dimensional image ensures the accuracy of the first coordinate system, laying a foundation for the measurement accuracy of the subsequent measurement process.

[0060] The method for generating an image measuring instrument measurement program based on a three-dimensional CAD model provided by an embodiment of the present application can be applied to, for example Figure 1In the application environment shown. Among them, the image measuring instrument may include a platform, a camera, and a controller. In some other embodiments, the image measuring instrument may further include an electric fixture with moving or rotating parts, or the platform may move or rotate parts. In some other embodiments, the image measuring instrument may include multiple cameras to have multiple shooting angles. The controller may store a measurement program obtained by template programming. The platform includes a precision linear slide rail and a servo control system to ensure system accuracy; and three-axis fully automatic programmable detection is realized to achieve batch detection of complex features. The camera is a high-resolution lens and has a large field of view, so as to balance measurement efficiency and accuracy; and it supports segmented programming control of surface light, transmitted light, and coaxial light; automatically identifies the measurement part, and can obtain a unified and stable measurement result each time.

[0061] Among them, template programming can be carried out on a terminal, etc. Open the imported 3D image on the terminal. The 3D image can be the spatial 3D CAD model of the part. Among them, the 3D image can be presented in the interface, and the user can rotate, move or select the features on the part in the interface, and then perform template programming to obtain a measurement program. Subsequently, the measurement program is imported into the controller of the image measuring instrument, so that the controller can execute the measurement method in this application based on the measurement program to realize the measurement of the part.

[0062] In some alternative embodiments, the image measuring instrument of the present application can be used in the fields of machinery, electronics, molds, injection molding, hardware, rubber, low-voltage electrical appliances, magnetic materials, precision stamping, connectors, terminals, mobile phones, household appliances, printed circuit boards, medical devices, watches, tools, metrology and testing, etc.

[0063] In some alternative embodiments, the image measuring instrument of the present application can achieve precise measurement of surface dimensions, contours, angles and positions, form and position tolerances, etc. of various complex parts.

[0064] In an exemplary embodiment, as Figure 2 shown, a method for generating a measurement program of an image measuring instrument based on a spatial 3D CAD model is provided. Taking the method applied to the Figure 1 terminal in it as an example for description, it includes the following steps S202 to step S208. Among them:

[0065] [[ID=IP19]]S202: Import the 3D image of the part, and the 3D image includes the spatial 3D CAD model.

[0066] Among them, the three-dimensional image is a three-dimensional part image, and the three-dimensional image may include a spatial three-dimensional CAD model. After importing the three-dimensional image of the part, the three-dimensional image of the part can be presented in the interface, and the user can rotate, move, or select part features on the part in the interface. That is to say, the terminal can receive instructions to rotate, move, or select part features on the part through the three-dimensional image, and rotate, move, or select part features on the part based on the instructions.

[0067] S204: Take the shooting direction determined based on the part features in the three-dimensional image as the first direction, determine the second direction according to the part features in the three-dimensional image, and fit each coordinate axis of the first coordinate system based on the first direction and the second direction.

[0068] S206: Determine the first origin of the first coordinate axis in the first coordinate system based on the part features in the three-dimensional image, and determine the origins of the other coordinate axes based on the part features for determining the first origin or other part features in the three-dimensional image.

[0069] Among them, the origin of each coordinate axis of the first coordinate system is determined based on the part features in the three-dimensional image. However, in order to improve the user experience, when defining the origin of each coordinate axis, the origin of the other coordinate axes can be determined based on the part features corresponding to the already determined origin or other part features. Other part features include the part features defined when determining each coordinate axis or newly defined part features, which are not specifically limited here.

[0070] Among them, the first coordinate system includes each coordinate axis and the origin of each coordinate axis. Each coordinate axis corresponds to each direction. In this application, each direction is determined based on the part features in the three-dimensional image, each coordinate axis of the first coordinate system is fitted based on each direction, and the origin of each coordinate axis can be determined based on the part features in the three-dimensional image.

[0071] The part features in the three-dimensional image can be selected by the user. That is to say, the terminal can receive part feature selection instructions through the three-dimensional image, then determine each part feature based on the part feature selection instructions, and determine each direction or the origin of each coordinate axis based on the selected part features. When determining the origin of each coordinate axis, the first origin of the first coordinate axis in the first coordinate system can be determined based on the part features in the three-dimensional image first, and then the origins of the other coordinate axes can be determined based on the part features for determining the first origin or other part features in the three-dimensional image. That is to say, for the subsequent determination of the coordinate origin of the coordinate axis, the part features corresponding to the already determined coordinate origin or newly selected part features by the user can be used. In this way, the user can select the existing part features as the reference without having to re-define part features in the three-dimensional image.

[0072] Optionally, the part features may include points, lines, surfaces, etc., which are not specifically limited herein.

[0073] Wherein the user can rotate the three-dimensional image to determine the position where the main features to be measured are located, and then determine the first direction and the second direction based on the positions where these main features are located.

[0074] In this application, the shooting direction can be determined based on the position where the main features are located. For example, the main features in the part features can be selected, and then the shooting direction is determined based on the main features. For example, the shooting direction is directly facing these main features, and this shooting direction is used as the first direction, and the first direction is the Z-axis direction or is basically consistent with the Z-axis direction.

[0075] In some alternative embodiments, when there are multiple main features and the shooting directions corresponding to the multiple main features are inconsistent, multiple first directions can be determined, thereby generating multiple first coordinate systems. Subsequently, corresponding measurement programs are generated based on each first coordinate system. These measurement programs can be stored separately or stored together, which is not specifically limited herein.

[0076] The second direction can be the X-axis direction or the Y-axis direction. In this application, for convenience, the second direction is taken as the X-axis direction. The user can select some features from the main features and determine the second direction based on these features.

[0077] After determining the first direction (Z-axis) and the second direction (X-axis), the third direction (corresponding to the Y-axis) perpendicular to the first direction and the second direction can be determined, that is, the XYZ-axis directions of the first coordinate system are obtained.

[0078] It should be noted that since this step is an artificial operation and there may not be perpendicular line segments in the three-dimensional image, the first direction and the second direction are not necessarily perpendicular. Therefore, the XYZ axes of the obtained first coordinate system are directions obtained by fitting based on the defined first direction and second direction, and the X-axis does not necessarily completely coincide with the original second direction.

[0079] S208: Generate a measurement program file based on the first coordinate system.

[0080] After determining the first coordinate system, that is, determining the shooting direction; in the case of multiple first coordinate systems, multiple shooting directions are obtained. Subsequently, the feature extraction method and the measurement method are respectively determined based on the first coordinate system to generate corresponding measurement program files.

[0081] In one of the optional embodiments, generating a measurement program based on the extraction method of the target feature, the measurement method of the target feature, and the first coordinate system includes: in the case where the image measuring instrument can only take pictures along one shooting direction, for different measurement planes, generating a measurement program based on the extraction method of the target feature, the measurement method of the target feature, and the first coordinate system; in the case where the image measuring instrument includes multiple shooting directions, storing the extraction method of the target feature, the measurement method of the target feature, and the first coordinate system corresponding to each different measurement plane into a measurement program.

[0082] Among them, if the image measuring instrument can only take pictures along one shooting direction, different measurement programs can be saved for different measurement planes, that is, different shooting directions. If the image measuring instrument has multiple shooting directions, for example, the image measuring instrument has an electric fixture or platform for moving or rotating the workpiece, so that multiple shooting directions can be realized, or has multiple shooting cameras with different shooting angles, then the extraction method of the target feature, the measurement method of the target feature, and the first coordinate system corresponding to each shooting direction are saved as one measurement program.

[0083] For the above measurement program generation method, import the three-dimensional image of the part; use the shooting direction determined based on the part features in the three-dimensional image as the first direction, and determine the second direction according to the part features in the three-dimensional image. Fit each coordinate axis of the first coordinate system based on the first direction and the second direction; determine the origin of each coordinate axis of each first coordinate system based on the part features in the three-dimensional image; generate a measurement program based on the first coordinate system. Among them, template programming can be directly performed in the three-dimensional image, and template programming can be performed based on different shooting directions, which improves the efficiency. Moreover, programming is directly based on the three-dimensional image file of the part, and the accuracy is higher. And establishing the first coordinate system based on this three-dimensional image ensures the accuracy of the first coordinate system, laying a foundation for the measurement accuracy of the subsequent measurement process.

[0084] In some optional embodiments, using the shooting direction determined based on the part features in the three-dimensional image as the first direction includes at least one of the following methods: determining the shooting direction based on the straight line selected in the three-dimensional image, and using the shooting direction as the first direction; determining the shooting direction based on the first plane selected in the three-dimensional image, and using the shooting direction as the first direction.

[0085] The first coordinate system includes three coordinate axes, namely the X-axis, the Y-axis, and the Z-axis. The first direction is by default the shooting direction of the camera of the image measuring instrument, that is, the Z-axis direction. In this way, the main features of the part can be obtained within the field of view of the image measuring instrument, which is convenient for subsequent measurement.

[0086] Optionally, the determination of the first direction can be defined by selecting a first plane or a straight line in the three-dimensional image, where the first plane is the plane where the main feature is located, or the main feature is near the first plane. The determination of the straight line is also because the main feature is near the straight line. It should be noted that, compared with the method of determining the first direction by selecting a straight line, the method of selecting a plane is beneficial to defining the origin of the first direction based on the first plane subsequently.

[0087] In some alternative embodiments, in combination with Figure 3 as shown, Figure 3 FIG. is a flowchart of the steps for determining the first direction in an embodiment, where the first direction is obtained based on the shooting direction, and the determination of the shooting direction, that is, the shooting direction is determined by selecting a first plane in the three-dimensional image, including: determining at least 3 non-collinear points in the part selected in the three-dimensional image; generating a first plane based on the at least 3 non-collinear points, and determining the normal vector direction of the first plane as the shooting direction.

[0088] Among them, the user can select at least 3 non-collinear points in the part in the three-dimensional image, so that a first plane can be generated based on the 3 non-collinear points, and thus the direction of the normal vector of the first plane is the shooting direction, that is, the first direction in the above text.

[0089] It should be noted that theoretically, an ideal plane can be fitted with 3 non-collinear selected points. However, in the actual measurement process, the plane of the solid workpiece is not necessarily an ideal plane. Therefore, selecting at least 4 selected points can improve the fitting accuracy of the first plane. That is, in the present application, at least 4 non-collinear points are arbitrarily selected in the three-dimensional image, and then a first plane is fitted based on the at least 4 non-collinear points, so that the direction of the normal vector of the first plane is the shooting direction, that is, the first direction in the above text.

[0090] In the above embodiments, since the user can freely rotate and move the three-dimensional image, the position of the main feature of the part can be determined, and then the corresponding part features are selected based on the position of the main feature to determine the first direction of the first coordinate system, that is, the shooting direction, so that the field of view during shooting can include the main feature of the part.

[0091] In one of the alternative embodiments, determining the second direction according to the part features in the three-dimensional image includes at least one of the following methods: fitting a first straight line based on at least two points selected in the three-dimensional image, and determining the direction of the first straight line as the second direction; using the straight line selected in the three-dimensional image as the first straight line, and determining the direction of the first straight line as the second direction.

[0092] The second direction corresponds to the X-axis direction or the Y-axis direction. Hereinafter, the case where the second direction corresponds to the X-axis direction will be described. In other embodiments, the second direction may be the Y-axis direction, so that the X-axis direction can be obtained by fitting based on the Z-axis direction and the Y-axis direction, and then the first coordinate system can be obtained.

[0093] Similarly, the second direction can be determined based on the selected part features, which may include points or lines. For example, the second direction can be defined by clicking multiple selected points or selecting a straight line.

[0094] The method of clicking multiple selected points means that at least two selected points can be selected in the three-dimensional image, and then a first straight line is fitted based on the at least two selected points, and the direction of the fitted first straight line is used as the second direction. Since the straight line feature can be directly selected in the three-dimensional image, the direction of the selected first straight line can be used as the second direction.

[0095] After defining the first direction (corresponding to the Z-axis) and the second direction (corresponding to the X-axis), a third direction (corresponding to the Y-axis) perpendicular to the first direction and the second direction can be defined. At this time, the XYZ-axis directions of the first coordinate system can be obtained. It should be noted that since this step is an artificial operation and there may not be perpendicular line segments in the three-dimensional image, the first direction and the second direction are not necessarily perpendicular. Therefore, after step S22, the XYZ axes of the obtained first coordinate system are the directions obtained by fitting based on the defined first direction and second direction, and the X-axis does not necessarily completely coincide with the original second direction.

[0096] In the above embodiments, the second direction is determined by selecting points or straight lines in the three-dimensional image. Since the three-dimensional image can be arbitrarily rotated, moved, and selected, the processing efficiency is improved.

[0097] In one optional embodiment, the first origin of the first coordinate axis in the first coordinate system is determined based on the part features in the three-dimensional image, and the origins of the other coordinate axes are determined based on the part features for determining the first origin or other part features in the three-dimensional image, including: obtaining the origin of the X-axis of the first coordinate system based on the projection of the first selected point selected in the three-dimensional image on the X-axis of the first coordinate system; generating a first option corresponding to the first selected point and a first option corresponding to the first straight line, and determining the origin of the Y-axis of the first coordinate system according to the first selection result of the first option corresponding to the first selected point and the first option corresponding to the first straight line, wherein when the first selection result is the first option corresponding to the first selected point, obtaining the origin of the Y-axis of the first coordinate system based on the projection of the first selected point on the Y-axis of the first coordinate system; when the first selection result is the first option corresponding to the first straight line, obtaining the origin of the Y-axis of the first coordinate system based on the projection of the first straight line on the Y-axis of the first coordinate system; generating a second option corresponding to the first selected point, a second option corresponding to the first straight line, and an option corresponding to the first plane, and determining the origin of the Z-axis of the first coordinate system according to the second selection result of the second option corresponding to the first selected point, the second option corresponding to the first straight line, and the option corresponding to the first plane, wherein when the second selection result is the second option corresponding to the first selected point, obtaining the origin of the Z-axis of the first coordinate system based on the projection of the first selected point on the Z-axis of the first coordinate system; when the second selection result is the second option corresponding to the first straight line, obtaining the origin of the Z-axis of the first coordinate system based on the projection of the first straight line on the Z-axis of the first coordinate system; when the second selection result is the option corresponding to the first plane, obtaining the origin of the Z-axis of the first coordinate system based on the projection of the first plane on the Z-axis of the first coordinate system.

[0098] Wherein, the first coordinate system includes coordinate axes and an origin. After determining the directions of the coordinate axes, it is also necessary to determine the origins of the coordinate axes, so that the first coordinate system can be used to label the coordinates of each feature in the part.

[0099] In this application, the origin can be defined based on the part features already selected by the user as a reference, without the user having to re-define new part features, and the part features already selected by the user can be determined when defining the origins of the other coordinate axes.

[0100] The origin of the X-axis can be determined based on the projection of the first selected point selected in the three-dimensional image on the X-axis of the first coordinate system. For example, the origin of the X-axis is defined by clicking on a selected point. The selected point clicked in this step is used as the first selected point, and the first selected point is used as the reference for the origin of the X-axis, that is, the projection position of the first selected point on the X-axis is the origin position of the X-axis.

[0101] When determining the origin of the Y-axis, the interface displays the first option corresponding to the first selected point and the first option corresponding to the first straight line, where the first straight line is a part feature determined when fitting the directions of the respective coordinate axes of the first coordinate axis. If the method of fitting the directions of the respective coordinate axes of the first coordinate axis does not use the first straight line but uses the first plane, then the first option corresponding to the first straight line is not displayed at this time. Subsequently, the terminal can receive the first selection result of the first option corresponding to the first selected point and the first option corresponding to the first straight line through the interface, and determine the origin of the Y-axis based on the first selection result. If the option corresponding to the first selected point is selected, the projection position of the first selected point on the Y-axis is the origin position of the Y-axis; if the option corresponding to the first straight line is selected, the projection position of the first straight line on the Y-axis is the origin position of the Y-axis. Whichever selection is made, the first coordinate system can be obtained, and the user can make a selection based on the relative position between the first coordinate system and the part. Here, the first straight line is the first straight line selected when defining the X-axis.

[0102] When determining the origin of the Z-axis, the interface displays the second option corresponding to the first selected point, the second option corresponding to the first straight line, and the selection corresponding to the first plane, where the first straight line and the first plane are part features determined when fitting the directions of the respective coordinate axes of the first coordinate axis. If the method of fitting the directions of the respective coordinate axes of the first coordinate axis does not use the first straight line but uses the first plane, then the first option corresponding to the first straight line is not displayed at this time. If the method of fitting the directions of the respective coordinate axes of the first coordinate axis does not use the first plane but uses the first straight line, then the first option corresponding to the first plane is not displayed at this time. Subsequently, the terminal can receive the second selection result of the second option corresponding to the first selected point, the second option corresponding to the first straight line, and the selection corresponding to the first plane through the interface, and determine the origin of the Y-axis based on the second selection result. In the case where the second selection result is the second option corresponding to the first selected point, the origin of the Z-axis of the first coordinate system is obtained based on the projection of the first selected point on the Z-axis of the first coordinate system; in the case where the second selection result is the second option corresponding to the first straight line, the origin of the Z-axis of the first coordinate system is obtained based on the projection of the first straight line on the Z-axis of the first coordinate system; in the case where the second selection result is the option corresponding to the first plane, the origin of the Z-axis of the first coordinate system is obtained based on the projection of the first plane on the Z-axis of the first coordinate system.

[0103] Here, the first straight line is the first straight line selected when defining the second direction. The first plane is the first plane selected when defining the first direction.

[0104] It should be noted that if the subsequent main features are all near the first plane, the first plane can be selected as the reference for the origin of the Z-axis.

[0105] In some alternative embodiments, generating a measurement program file based on a first coordinate system includes: determining an extraction method for a target feature in a part based on the first coordinate system and a three-dimensional image; determining a measurement method for the target feature in the three-dimensional image of the part; and generating a measurement program file based on the extraction method for the target feature, the measurement method for the target feature, and the first coordinate system.

[0106] The extraction method for the target feature includes at least one of a feature point extraction method, a feature line extraction method, a feature circle extraction method, and a feature plane extraction method.

[0107] The measurement method for the target feature includes at least one of a target feature length measurement method, an angle measurement method, and a height measurement method.

[0108] After determining the extraction method for the target feature and the measurement method for the target feature, a measurement program file can be obtained.

[0109] In some alternative embodiments, the order of determining the extraction method for the target feature and the measurement method for the target feature can be swapped, that is, first select the measurement method for the target feature, and then determine the extraction method for the target feature. No specific limitation is made on the order of the two here.

[0110] In some alternative embodiments, determining an extraction method for a target feature in a part based on a first coordinate system and a three-dimensional image includes: performing a simulation based on the first coordinate system and the three-dimensional image to obtain a simulation image when shooting along the shooting direction; and determining the extraction method for the target feature in the part based on the simulation image and the three-dimensional image.

[0111] After the first coordinate system is determined, since the Z-axis direction has been set, a simulation image obtained by simulating the shooting of the actual workpiece can be based on the three-dimensional image. This simulation image is a binary image, and this binary image is binarized based on height, thereby being able to reduce interference information and improve the clarity of contour information. Thus, it is convenient for the user to know the approximate position of the current selection area or scan line in the image of the actual workpiece when programming on the three-dimensional image.

[0112] In other embodiments, if there are multiple first coordinate systems, simulation can be performed respectively based on each first coordinate system to obtain respective simulation images when the actual workpiece is shot along the Z-axis of the corresponding first coordinate system.

[0113] In one of the alternative embodiments, the extraction method of the target feature includes at least one of a feature point extraction method, a feature line extraction method, a feature circle extraction method, and a feature plane extraction method; the feature point extraction method includes at least one of a method of determining feature points based on manual selection and a method of determining feature points based on scanning, wherein the method of determining feature points based on scanning includes determining two first candidate points selected in the three-dimensional image, and scanning the boundary between the two first candidate points to obtain each feature point; the feature line extraction method includes: determining at least two second candidate points selected in the three-dimensional image, and based on the at least two second candidate points, displaying a first point selection area in the simulation image, and performing fitting based on each feature point selected in the first point selection area to obtain a feature line; the feature circle extraction method includes: determining a feature circle selected in the three-dimensional image, and obtaining a target feature circle based on the selected feature circle; the feature plane extraction method includes: determining a plane selected in the three-dimensional image as a feature plane.

[0114] For the convenience of understanding, the above-mentioned feature point extraction method, feature line extraction method, feature circle extraction method, and feature plane extraction method will be described separately.

[0115] Among them, the feature point extraction method is the basis for other extraction methods, and this feature point extraction method includes at least one of a method of determining feature points based on manual selection and a method of determining feature points based on scanning.

[0116] Among them, the method of determining feature points based on manual selection is to directly pick points, that is, the terminal receives a point selection instruction through the three-dimensional image, and performs feature point recognition based on the position corresponding to the point selection instruction. Specifically, the user directly clicks on the three-dimensional image to select points, and the subsequent program will perform feature point recognition at this position.

[0117] The method of determining feature points based on scanning is that the method of determining feature points based on scanning includes determining two first candidate points selected in the three-dimensional image, and scanning the boundary between the two first candidate points to obtain each feature point. Specifically, the user directly clicks two points on the three-dimensional image, and when the subsequent execution program is running, it can recognize the boundary between the two selected points, and then recognize the feature points located on the boundary. Among them, the method of recognizing the boundary can be to calculate the contrast of each position in the selected point connection line, and select the position with the largest contrast as the boundary between the connection lines.

[0118] The method for extracting the feature line may include the terminal receiving a selection instruction for the second candidate point through a three-dimensional image, then determining at least two second candidate points in the three-dimensional image based on the selection instruction, and further displaying a first point-taking area in the simulation image based on the at least two second candidate points. When the subsequent program is executed, the program identifies multiple feature points in the corresponding first point-taking area in the image, and fits the multiple feature points to obtain the feature line. In this embodiment, the first point-taking area may be a rectangular area, and multiple scan lines perpendicular to the connection direction of the second candidate points may be set in the rectangular area, and the boundaries of each scan line are identified, and then the feature points located at the boundaries are identified.

[0119] The method for extracting the feature circle includes: the terminal receiving a feature circle selection instruction through a three-dimensional image, and then obtaining a target feature circle based on the selected feature circle, which may specifically include at least one of methods such as sector point-taking, annular point-taking, multi-point point-taking, and scan line point-taking.

[0120] In some alternative embodiments, obtaining a target feature circle based on the selected feature circle includes any of the following methods: generating a second point-taking area based on the feature circle, and fitting the feature points identified in the second point-taking area to obtain the feature circle; where the second point-taking area is a sector point-taking area or an annular point-taking area; identifying a point set at the junction based on multiple selected points selected near the feature circle, and fitting the point set to obtain the target feature circle; obtaining the number of set scan lines, generating each scan line passing through the feature circle in the simulation image, generating each feature point based on the scan line, and fitting the generated each feature point to obtain the target feature circle.

[0121] Among them, for sector point-taking, after the user clicks on the feature circle in the three-dimensional image, that is, after the terminal receives the feature circle selection instruction through the three-dimensional image, a second point-taking area in the shape of a fan ring is formed near the feature circle in the simulated simulation image. When the subsequent program is executed, the program will identify the point set at the black-and-white junction in the second point-taking area, fit the point set to obtain the feature circle, and thus be able to confirm the position of the target feature circle on the part.

[0122] The difference between annular point-taking and sector point-taking is that the second point-taking area corresponding to annular point-taking is a ring. Among them, the width of the ring can be adjusted, thereby being able to expand or shrink the size of the second point-taking area. Expanding the width of the ring can avoid the feature circle on the actual part deviating from the designed position and not being able to identify the feature circle in the image during the execution of the program, and shrinking the width of the ring can improve the accuracy of identifying the feature circle during the execution of the program.

[0123] Multi-point selection means that the user clicks multiple selection points near the feature circle in the three-dimensional image, that is, receives a selection point instruction. The selection point instruction is used for the terminal to obtain multiple selection points near the feature circle. The area near the feature circle can be understood as the distance between these selection points and the feature circle within the preset distance range. When the subsequent program is executed, the program will identify the point set at the black-and-white junction near the selection points, and fit the feature circle through the point set.

[0124] Scan line selection means that after the user clicks on the feature circle in the three-dimensional image, the number of scan lines is set. The system will generate multiple scan lines in the simulated simulation image. Optionally, these multiple scan lines pass through the center of the feature circle and are symmetrically distributed around the center. When the subsequent program is executed, the program obtains the feature points in each scan line based on the scan line selection in the above-mentioned extracted feature points, and then fits a circle based on the feature points in the scan lines.

[0125] The feature plane extraction method includes receiving a plane selection instruction through the three-dimensional image, and using the plane selected in the three-dimensional image based on this selection instruction as the feature plane.

[0126] In the above embodiments, the determination process of the extraction method of each target feature is given. The user can directly operate in the three-dimensional image to determine the extraction method of each target feature, which is more intuitive and improves the processing efficiency.

[0127] In one optional embodiment, determining the measurement method of the target feature in the part in the three-dimensional image includes: determining at least one of the length measurement method, angle measurement method, and height measurement method of the target feature in the part in the three-dimensional image; where the length measurement method is used to measure the length of the target feature; the angle measurement method is used to measure the angle of the target feature; the height measurement method includes measuring the height corresponding to the target feature based on a laser ranging device or measuring the height of the target feature based on a camera. Measuring the height of the target feature based on a camera includes: moving the camera above the target feature and moving the camera along the Z-axis. When the quality of the image collected by the camera meets the image quality requirements, the height of the camera is used as the height of the target feature.

[0128] Among them, after or before programming the extraction method of the target feature, programming the measurement method of the target feature for analyzing at least one of the length, angle, and height can be performed. For example, a ranging tool can be selected, and after selecting the corresponding target feature, the distance between the target features can be displayed. When the subsequent program is executed, when the program completes the extraction of the target feature, the same analysis actions for at least one of the length, angle, and height will also be executed, so as to directly determine whether the machining of the part meets the design requirements.

[0129] For the height measurement method, points can be selected by clicking in the three-dimensional image. The terminal receives the point selection instruction and determines the corresponding feature points based on the point selection instruction. When the subsequent program is executed, the lens of the image measuring instrument can move above the position corresponding to the selected feature point in the part and move the lens along the Z-axis to record the height when the field of view is clearest. When measuring the height in this way, the magnification of the lens can be adjusted until the lens has a small depth of field. Therefore, when the lens moves to near the height where the field of view is clearest, due to the small depth of field of the lens of the image measuring instrument, the field of view can quickly become blurred or clear, and thus the height can be determined more accurately.

[0130] In addition, for an image measuring instrument equipped with a laser ranging device, the height of the position corresponding to the selected feature point in the part can also be directly measured by the laser ranging device.

[0131] During the template programming process, multiple tools can be used to identify different target features, and then the lengths, angles, and heights of the features between different target features can be measured through analysis tools, so as to comprehensively detect the part.

[0132] In an exemplary embodiment, as Figure 4 shown, a measurement method is provided. Taking the image measuring instrument in Figure 1 as an example for illustration, it includes the following steps S402 to step S406. Among them:

[0133] S402: Obtain the captured image to be processed.

[0134] This embodiment is the step of officially starting the automatic measurement. The image measuring instrument captures the part image in real time to obtain the image to be processed, and this image to be processed is used for feature extraction.

[0135] In some optional embodiments, the image to be processed can be an image captured by the image measuring instrument (for example, an image under an illumination system such as bottom lighting, coaxial light, or ring light), or a binary image obtained after processing. This binary image corresponds to the simulation image in the above text, except that the simulation image is obtained by simulation.

[0136] S404: Establish a second coordinate system based on the image to be processed.

[0137] The second coordinate system is established based on the image to be processed. The second coordinate system can completely coincide with the first coordinate system or not completely coincide, and specific limitations are not made here. This is because the measurement program can identify each feature within a relatively large range. Therefore, it is not necessary to limit the complete coincidence of the second coordinate system and the first coordinate system, which can reduce the user's difficulty of use.

[0138] The establishment method of the second coordinate system includes two types: automatic establishment and manual establishment. Compared with the method of automatically establishing the second coordinate system, manual establishment of the second coordinate system is more suitable for workpieces with complex shapes or difficult to directly identify the posture according to the contour.

[0139] In one optional embodiment, an example of an automatic establishment method is given. When the measurement program is stored, the overall contour of the part in the shooting direction is stored; establishing the second coordinate system based on the image to be processed includes: identifying the posture of the part in the image to be processed based on the overall contour; generating the second coordinate system based on the recognition result of the posture and the first coordinate system corresponding to the shooting direction.

[0140] In this embodiment, when saving the automatic measurement program, the approximate overall contour of the three-dimensional image in the shooting direction can be saved. When the program is run subsequently, the posture of the workpiece in the image to be processed can be identified based on the approximate overall contour, so as to determine the first coordinate system corresponding to the contour with the highest similarity to the image to be processed, and then generate the second coordinate system based on this first coordinate system. When automatically generated, the second coordinate system basically coincides with the first coordinate system.

[0141] In one optional embodiment, an example of a manual establishment method is given. The method of manually establishing the second coordinate system is similar to the method of establishing the first coordinate system. Specifically, establishing the second coordinate system based on the image to be processed includes: determining the fourth direction and the fifth direction according to the part features in the image to be processed, determining the sixth direction according to the shooting direction, and fitting the coordinate axes of the second coordinate system based on the fourth direction, the fifth direction and the sixth direction; determining the origin of each coordinate axis of the second coordinate system based on the part features in the image to be processed; determining the second coordinate system based on the coordinate axes of the second coordinate system and the origin of each coordinate axis of the second coordinate system.

[0142] The part is placed in the field of view of the image measuring instrument, and then an image is taken to obtain the image to be processed. Two directions are determined on the image to be processed, one is the fourth direction, and the other is the fifth direction. The fourth direction and the fifth direction respectively correspond to the X-axis direction and the Y-axis direction of the second coordinate system, and the shooting direction is defaulted to the Z-axis direction of the second coordinate system. Thus, the second coordinate system can be fitted based on the two selected directions and the shooting direction. The position of the Z-axis origin can be located on the plane that can be clearly displayed in the image measuring instrument.

[0143] S406: Measuring the part in the image to be processed based on the measurement program and the second coordinate system, and the measurement program is generated based on the measurement program generation method in any of the above embodiments.

[0144] After determining the second coordinate system, the subsequent measurement program can identify each feature within a relatively large range and calculate information such as the size corresponding to each feature.

[0145] In one of the optional embodiments, in combination with Figure 5 as shown Figure 5 is a schematic flow chart of the measurement method in an embodiment. First, a three-dimensional image is imported. The three-dimensional image can be presented in the interface, and the user can rotate, move, or select features on the part in the interface.

[0146] Secondly, a first coordinate system is defined based on the three-dimensional image. The determination process of the first coordinate system includes: taking the shooting direction determined based on the part features in the three-dimensional image as the first direction, and determining the second direction according to the part features in the three-dimensional image. Each coordinate axis of the first coordinate system is obtained by fitting based on the first direction and the second direction; the origin of each coordinate axis of the first coordinate system is determined based on the part features in the three-dimensional image.

[0147] Thirdly, the extraction method of the target feature is determined.

[0148] Fourthly, the measurement method of the target feature is determined.

[0149] Fifthly, the measurement program file is saved.

[0150] Sixthly, the imaging measuring instrument takes a picture of the part to obtain an image to be processed.

[0151] Seventhly, a second coordinate system is defined, and the saved measurement program file is executed.

[0152] In the above measurement method, template programming is directly performed in the three-dimensional image. Template programming can be performed based on different shooting directions, which improves the efficiency. Moreover, programming is directly based on the three-dimensional image file of the part, with higher accuracy. And the establishment of the first coordinate system based on the three-dimensional image ensures the accuracy of the first coordinate system, laying a foundation for the measurement accuracy of the subsequent measurement process.

[0153] It should be understood that although the steps in the flow charts involved in the above embodiments are sequentially shown according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless there is a clear description in this article, the execution of these steps has no strict order limit, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flow charts involved in the above embodiments may include multiple steps or multiple stages. These steps or stages are not necessarily executed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be executed alternately or alternately with at least a part of other steps or steps in other steps.

[0154] Based on the same inventive concept, the embodiments of the present application further provide a measurement program generation device for implementing the measurement program generation method involved above, as well as a measurement device corresponding to the measurement method. The solution provided by this device to solve the problem is similar to the solution described in the above method. Therefore, the specific limitations in one or more embodiments of the measurement program generation device and the measurement device provided below can refer to the limitations on the measurement program generation method and the measurement method in the above text, and will not be repeated here.

[0155] In an exemplary embodiment, as Figure 6 shown, a measurement program generation device for an image measuring instrument based on a three-dimensional CAD model of space is provided, including: an import module 601, a first coordinate system generation module 602, and a measurement program generation module 603, where:

[0156] The import module 601 is configured to import a three-dimensional image of a part, and the three-dimensional image includes a three-dimensional CAD model of space;

[0157] The first coordinate system generation module 602 is configured to use the shooting direction determined based on the part features in the three-dimensional image as the first direction, and determine a second direction based on the part features in the three-dimensional image. Each coordinate axis of the first coordinate system is obtained by fitting based on the first direction and the second direction; the first origin of the first coordinate axis in the first coordinate system is determined based on the part features in the three-dimensional image, and the origins of the other coordinate axes are determined based on the part features for determining the first origin or other part features in the three-dimensional image;

[0158] The measurement program generation module 603 is configured to generate a measurement program file based on the first coordinate system.

[0159] In one optional embodiment, the above first coordinate system generation module 602 is further configured to determine the first direction based on at least one of the following methods: determining the shooting direction based on a straight line selected in the three-dimensional image, and using the shooting direction as the first direction; determining the shooting direction based on a first plane selected in the three-dimensional image, and using the shooting direction as the first direction.

[0160] In one optional embodiment, the above first coordinate system generation module 602 is further configured to determine at least three non-collinear points in the part selected in the three-dimensional image; generate a first plane based on the at least three non-collinear points, and determine the normal vector direction of the first plane as the shooting direction.

[0161] In one optional embodiment, the above-mentioned first coordinate system generation module 602 is further configured to determine the second direction based on at least one of the following methods: fitting a first straight line based on at least two points selected in the three-dimensional image, and determining the direction of the first straight line as the second direction; using the straight line selected in the three-dimensional image as the first straight line, and determining the direction of the first straight line as the second direction.

[0162] In one optional embodiment, the above-mentioned first coordinate system generation module 602 is further configured to: obtain the origin of the X-axis of the first coordinate system based on the projection of the first selected point selected in the three-dimensional image on the X-axis of the first coordinate system; generate a first option corresponding to the first selected point and a first option corresponding to the first straight line, and determine the origin of the Y-axis of the first coordinate system according to the first selection result of the first option corresponding to the first selected point and the first option corresponding to the first straight line, wherein when the first selection result is the first option corresponding to the first selected point, obtaining the origin of the Y-axis of the first coordinate system based on the projection of the first selected point on the Y-axis of the first coordinate system; when the first selection result is the first option corresponding to the first straight line, obtaining the origin of the Y-axis of the first coordinate system based on the projection of the first straight line on the Y-axis of the first coordinate system; generating a second option corresponding to the first selected point, a second option corresponding to the first straight line, and an option corresponding to the first plane, and determining the origin of the Z-axis of the first coordinate system according to the second selection result of the second option corresponding to the first selected point, the second option corresponding to the first straight line, and the option corresponding to the first plane, wherein when the second selection result is the second option corresponding to the first selected point, obtaining the origin of the Z-axis of the first coordinate system based on the projection of the first selected point on the Z-axis of the first coordinate system; when the second selection result is the second option corresponding to the first straight line, obtaining the origin of the Z-axis of the first coordinate system based on the projection of the first straight line on the Z-axis of the first coordinate system; when the second selection result is the option corresponding to the first plane, obtaining the origin of the Z-axis of the first coordinate system based on the projection of the first plane on the Z-axis of the first coordinate system.

[0163] In one optional embodiment, the above-mentioned measurement program generation module 603 is further configured to: determine the extraction method of the target feature in the part based on the first coordinate system and the three-dimensional image; determine the measurement method of the target feature in the part in the three-dimensional image; generate a measurement program file based on the extraction method of the target feature, the measurement method of the target feature, and the first coordinate system.

[0164] In one optional embodiment, the above-mentioned measurement program generation module 603 is further configured to: perform simulation based on the first coordinate system and the three-dimensional image to obtain a simulation image when shooting along the shooting direction; determine the extraction method of the target feature in the part based on the simulation image and the three-dimensional image.

[0165] In one of the alternative embodiments, the extraction method of the target feature includes at least one of a feature point extraction method, a feature line extraction method, a feature circle extraction method, and a feature plane extraction method;

[0166] The feature point extraction method includes at least one of a method of determining feature points based on manual selection and a method of determining feature points based on scanning. The method of determining feature points based on scanning includes determining two first candidate points selected in the three-dimensional image and scanning the boundary between the two first candidate points to obtain each feature point;

[0167] The feature line extraction method includes: determining at least two second candidate points selected in the three-dimensional image, displaying a first point selection area in the simulation image based on the at least two second candidate points, and performing fitting based on each feature point selected in the first point selection area to obtain a feature line;

[0168] The feature circle extraction method includes: determining a feature circle selected in the three-dimensional image and obtaining a target feature circle based on the selected feature circle;

[0169] The feature plane extraction method includes: determining a plane selected in the three-dimensional image as a feature plane.

[0170] In one of the alternative embodiments, the above measurement program generation module 603 is further configured to determine a target feature circle based on any one of the following methods: generating a second point selection area based on the feature circle, and performing fitting based on the feature points identified in the second point selection area to obtain a feature circle; where the second point selection area is a sector point selection area or an annular point selection area; identifying a point set at the junction based on a plurality of selected points selected near the feature circle, and performing fitting based on the point set to obtain a target feature circle; obtaining the number of set scan lines, generating each scan line passing through the feature circle in the simulation image, generating each feature point based on the scan lines, and performing fitting based on the generated feature points to obtain a target feature circle.

[0171] In one of the alternative embodiments, the above measurement program generation module 603 is further configured to determine at least one of a target feature length measurement method, an angle measurement method, and a height measurement method for a part in the three-dimensional image; where the length measurement method is used to measure the length of the target feature; the angle measurement method is used to measure the angle of the target feature; the height measurement method includes measuring the height corresponding to the target feature based on a laser ranging device or measuring the height of the target feature based on a camera. Measuring the height of the target feature based on a camera includes: moving the camera above the target feature and moving the camera along the Z-axis, and taking the height of the camera when the quality of the image collected by the camera meets the image quality requirements as the height of the target feature.

[0172] In one optional embodiment, the above-mentioned measurement program generation module 603 is further configured to generate a measurement program for different measurement surfaces based on the extraction method of the target feature, the measurement method of the target feature, and the first coordinate system when the image measuring instrument can only take pictures along one shooting direction; when the image measuring instrument includes multiple shooting directions, store the extraction method of the target feature, the measurement method of the target feature, and the first coordinate system corresponding to each different measurement surface in a measurement program.

[0173] In an exemplary embodiment, as Figure 7 shown, a full-automatic measurement device for an image measuring instrument based on a spatial three-dimensional CAD model is provided, including: an image acquisition module 701, a second coordinate system generation module 702, and a measurement module 703, where:

[0174] The image acquisition module 701 is configured to acquire the to-be-processed image taken;

[0175] The second coordinate system generation module 702 is configured to establish a second coordinate system based on the to-be-processed image;

[0176] The measurement module 703 is configured to measure the parts in the to-be-processed image based on the measurement program and the second coordinate system, and the measurement program is generated by the measurement program generation device in any of the above embodiments.

[0177] In one optional embodiment, the overall contour of the part in the shooting direction is stored when the measurement program is stored; the above-mentioned second coordinate system generation module 702 is further configured to identify the pose of the part in the to-be-processed image based on the overall contour; generate a second coordinate system based on the recognition result of the pose and the first coordinate system corresponding to the shooting direction.

[0178] In one optional embodiment, the above-mentioned second coordinate system generation module 702 is further configured to determine a fourth direction and a fifth direction according to the part features in the to-be-processed image, determine a sixth direction according to the shooting direction, and fit each coordinate axis of the second coordinate system based on the fourth direction, the fifth direction, and the sixth direction; determine the origin of each coordinate axis of the second coordinate system based on the part features in the to-be-processed image; determine the second coordinate system based on each coordinate axis of the second coordinate system and the origin of each coordinate axis of the second coordinate system.

[0179] Each module in the above-mentioned measurement program generation device for an image measuring instrument based on a spatial three-dimensional CAD model and the full-automatic measurement device for an image measuring instrument based on a spatial three-dimensional CAD model can be implemented in whole or in part by software, hardware, and their combination. Each of the above modules can be embedded in or independent of the processor in the computer device in the form of hardware, or stored in the memory in the computer device in the form of software, so that the processor can call and execute the operations corresponding to each of the above modules.

[0180] In an exemplary embodiment, a computer device is provided. The computer device may be a terminal, and its internal structure diagram may be as shown in Figure 8 . The computer device includes a processor, a memory, an input / output interface, a communication interface, a display unit, and an input device. Among them, the processor, the memory, and the input / output interface are connected through a system bus, and the communication interface, the display unit, and the input device are connected to the system bus through the input / output interface. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The input / output interface of the computer device is used to exchange information between the processor and external devices. The communication interface of the computer device is used to communicate with external terminals in a wired or wireless manner. The wireless manner can be implemented through WIFI, a mobile cellular network, near field communication (NFC), or other technologies. When the computer program is executed by the processor, it implements a method for generating a measurement program of an image measuring instrument based on a three-dimensional CAD model in space and a fully automatic measurement method of an image measuring instrument based on a three-dimensional CAD model in space. The display unit of the computer device is used to form a visually visible picture, which can be a display screen, a projection device, or a virtual reality imaging device. The display screen can be a liquid crystal display screen or an electronic ink display screen. The input device of the computer device can be a touch layer covered on the display screen, or a button, a trackball, or a touchpad provided on the computer device housing, or an external keyboard, touchpad, or mouse, etc.

[0181] Those skilled in the art can understand that Figure 8 the structure shown in is only a block diagram of some structures related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine some components, or have a different component layout.

[0182] In an embodiment, a computer device is further provided, including a memory and a processor. A computer program is stored in the memory, and when the processor executes the computer program, the steps in the above method embodiments are implemented.

[0183] In an embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by the processor, the steps in the above method embodiments are implemented.

[0184] In one embodiment, a computer program product is provided, including a computer program which, when executed by a processor, implements the steps in the above method embodiments.

[0185] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use, and processing of relevant data need to comply with relevant regulations.

[0186] Those of ordinary skill in the art can understand that all or part of the processes in the above method embodiments can be completed by instructing relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the above method embodiments. Among them, any reference to a memory, database, or other medium used in the embodiments provided in this application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include Read-Only Memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, Resistive Random Access Memory (ReRAM), Magnetoresistive Random Access Memory (MRAM), Ferroelectric Random Access Memory (FRAM), Phase Change Memory (PCM), graphene memory, etc. Volatile memory can include Random Access Memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can be in various forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM), etc. The databases involved in the embodiments provided in this application can include at least one of relational databases and non-relational databases. Non-relational databases can include distributed databases based on blockchain, etc., without limitation. The processors involved in the embodiments provided in this application can be general-purpose processors, central processors, graphics processors, digital signal processors, programmable logic devices, data processing logics based on quantum computing, Artificial Intelligence (AI) processors, etc., without limitation.

[0187] The technical features of the above embodiments can be combined arbitrarily. For the sake of concise description, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, it should be considered as the scope recorded in this application.

[0188] The above-described embodiments merely represent several implementation manners of this application. The description is relatively specific and detailed, but it should not be construed as a limitation on the patent scope of this application. It should be noted that for those of ordinary skill in the art, without departing from the concept of this application, several modifications and improvements can still be made, and these all belong to the protection scope of this application. Therefore, the protection scope of this application shall be subject to the appended claims.

Claims

1. A fully automatic measurement method for an image measuring instrument based on a three-dimensional CAD model in space, characterized in that, The method includes: Importing a three-dimensional image of a part, where the three-dimensional image includes a spatial three-dimensional CAD model; Taking the shooting direction determined based on the part features in the three-dimensional image as the first direction, determining a second direction according to the part features in the three-dimensional image, and fitting each coordinate axis of the first coordinate system based on the first direction and the second direction; Determining the first origin of the first coordinate axis in the first coordinate system based on the part features in the three-dimensional image, and determining the origins of other coordinate axes based on the part features for determining the first origin or other part features in the three-dimensional image; Generating a measurement program file based on the first coordinate system; Obtaining a captured image to be processed, and establishing a second coordinate system based on the image to be processed; Measuring the part in the image to be processed based on the measurement program and the second coordinate system; The taking the shooting direction determined based on the part features in the three-dimensional image as the first direction includes at least one of the following methods: Determining the shooting direction based on a selected straight line in the three-dimensional image, and taking the shooting direction as the first direction; Determining the shooting direction based on a selected first plane in the three-dimensional image, and taking the shooting direction as the first direction, including: determining at least 3 non-collinear points in the part selected in the three-dimensional image; generating a first plane based on the at least 3 non-collinear points, and determining the normal vector direction of the first plane as the shooting direction; The determining the second direction according to the part features in the three-dimensional image includes at least one of the following methods: Fitting a first straight line based on at least two selected points in the three-dimensional image, and determining the direction of the first straight line as the second direction; Taking a selected straight line in the three-dimensional image as the first straight line, and determining the direction of the first straight line as the second direction; The determining the first origin of the first coordinate axis in the first coordinate system based on the part features in the three-dimensional image, and determining the origins of other coordinate axes based on the part features for determining the first origin or other part features in the three-dimensional image includes: Obtaining the origin of the X-axis of the first coordinate system based on the projection of a first selected point selected in the three-dimensional image on the X-axis of the first coordinate system; Generating a first option corresponding to the first selected point and a first option corresponding to the first straight line, and determining the origin of the Y-axis of the first coordinate system according to the first selection result of the first option corresponding to the first selected point and the first option corresponding to the first straight line, where in the case where the first selection result is the first option corresponding to the first selected point, obtaining the origin of the Y-axis of the first coordinate system based on the projection of the first selected point on the Y-axis of the first coordinate system; in the case where the first selection result is the first option corresponding to the first straight line, obtaining the origin of the Y-axis of the first coordinate system based on the projection of the first straight line on the Y-axis of the first coordinate system; Generate the second option corresponding to the first selected point, the second option corresponding to the first straight line, and the option corresponding to the first plane, and determine the origin of the Z-axis of the first coordinate system according to the second selection results of the second option corresponding to the first selected point, the second option corresponding to the first straight line, and the option corresponding to the first plane. Wherein, when the second selection result is the second option corresponding to the first selected point, based on the first selected point, the origin of the Z-axis of the first coordinate system is obtained by projecting onto the Z-axis of the first coordinate system; when the second selection result is the second option corresponding to the first straight line, the origin of the Z-axis of the first coordinate system is obtained by projecting the first straight line onto the Z-axis of the first coordinate system; when the second selection result is the option corresponding to the first plane, the origin of the Z-axis of the first coordinate system is obtained by projecting the first plane onto the Z-axis of the first coordinate system; The establishing of the second coordinate system based on the to-be-processed image includes: Determine the fourth direction and the fifth direction according to the part features in the to-be-processed image, determine the sixth direction according to the shooting direction, and fit each coordinate axis of the second coordinate system based on the fourth direction, the fifth direction, and the sixth direction; Determine the origin of each coordinate axis of each of the second coordinate systems based on the part features in the to-be-processed image; Determine the second coordinate system based on each coordinate axis of the second coordinate system and the origin of each coordinate axis of the second coordinate system.

2. The method according to claim 1, wherein The generating of the measurement program file based on the first coordinate system includes: Determine the extraction method of the target feature in the part based on the first coordinate system and the three-dimensional image, including: performing simulation based on the first coordinate system and the three-dimensional image to obtain a simulation image when shooting along the shooting direction; determining the extraction method of the target feature in the part based on the simulation image and the three-dimensional image; Determine the measurement method of the target feature in the part in the three-dimensional image; Generate a measurement program file based on the extraction method of the target feature, the measurement method of the target feature, and the first coordinate system.

3. The method according to claim 2, characterized in that The extraction method of the target feature includes at least one of a feature point extraction method, a feature straight line extraction method, a feature circle extraction method, and a feature plane extraction method; The feature point extraction method includes at least one of a method of determining feature points based on manual selection and a method of determining feature points based on scanning. Wherein, the method of determining feature points based on scanning includes determining two first candidate points selected in the three-dimensional image, and scanning the boundary between the two first candidate points to obtain each feature point; The feature straight line extraction method includes: determining at least two second candidate points selected in the three-dimensional image, and based on the at least two second candidate points, displaying a first point selection area in the simulation image, and performing fitting based on each feature point selected in the first point selection area to obtain a feature straight line; The feature circle extraction method includes: determining a feature circle selected in the three-dimensional image, and obtaining a target feature circle based on the selected feature circle, including any of the following methods: generating a second point selection area based on the feature circle, and fitting a feature circle based on the feature points identified in the second point selection area, where the second point selection area is a sector point selection area or an annular point selection area; identifying a point set at the junction based on a plurality of selected points selected near the feature circle, and fitting a target feature circle based on the point set; and obtaining the number of set scan lines, generating each scan line passing through the feature circle in the simulation image, generating each feature point based on the scan lines, and fitting a target feature circle based on the generated feature points. The feature plane extraction method includes: determining a plane selected in the three-dimensional image as the feature plane.

4. The method according to claim 2, characterized in that The method for determining the measurement method of the target feature in the part in the three-dimensional image includes: determining at least one of the length measurement method, the angle measurement method, and the height measurement method of the target feature in the part in the three-dimensional image; wherein the length measurement method is used to measure the length of the target feature; the angle measurement method is used to measure the angle of the target feature; the height measurement method includes measuring the height corresponding to the target feature based on a laser ranging device or measuring the height of the target feature based on a camera. Measuring the height of the target feature based on a camera includes: moving the camera above the target feature and moving the camera along the Z-axis, and taking the height of the camera when the quality of the image collected by the camera meets the image quality requirement as the height of the target feature.

5. The method according to claim 2, wherein Generating a measurement program based on the extraction method of the target feature, the measurement method of the target feature, and the first coordinate system includes: In the case where the image measuring instrument can only take pictures along one shooting direction, generating a measurement program based on the extraction method of the target feature, the measurement method of the target feature, and the first coordinate system for different measurement surfaces; In the case where the image measuring instrument includes multiple shooting directions, storing the extraction method of the target feature, the measurement method of the target feature, and the first coordinate system corresponding to each different measurement surface in a measurement program.

6. The method according to claim 1, wherein When the measurement program is stored, the overall contour of the part in the shooting direction is stored; establishing a second coordinate system based on the image to be processed includes: identifying the posture of the part in the image to be processed based on the overall contour; generating a second coordinate system based on the recognition result of the posture and the first coordinate system corresponding to the shooting direction.

Images