Photographing and measuring method, device, equipment and storage medium

Abstract

An embodiment of the present disclosure relates to a photography measurement method, device, equipment, and storage medium, which obtains multiple sets of synchronized images of a measured object having multiple marker points on its surface, which are successively photographed by a compound eye camera, extracts the coordinates of the image points corresponding to the marker points in each set of synchronized images, and reconstructs first 3D coordinates of the marker points corresponding to the image points based on the calibration data of the compound eye camera and the coordinates of the image points to obtain multiple sets of 3D marker points, and constructs a marker point global framework corresponding to the marker points on the surface of the measured object based on the multiple sets of 3D marker points, thereby enabling photography measurement to be realized without setting coding points or with only very few coding points, thereby reducing the manual workload and improving measurement efficiency and accuracy.

Description

[Technical field]

[0001] This disclosure claims priority to a Chinese patent application filed with the China Patent Office on September 27, 2021, bearing application number 202111133068.3 and titled "Photography measurement method, apparatus, equipment and storage medium," the entire contents of which are incorporated herein by reference. The embodiments of the present disclosure relate to the field of photography and measurement technology, and in particular to a photography and measurement method, device, apparatus, and storage medium. [Background technology]

[0002] Currently, in the photographic measurement realized by a monocular camera, it is generally necessary to manually set a large number of coding points on the surface of the measured object, and the process of setting requires some experience, such as the density distribution of the setting, the spatial position relationship, etc. After the photographic measurement is completed, these coding points need to be collected, and the process of setting and collecting takes a very long time. In addition, if the position of the coding points is moved during the photographic process, it will cause measurement failure or a decrease in measurement accuracy. Therefore, in order to solve the above problems, a new photographic measurement method is strongly required. Summary of the Invention

[0003] The embodiments of the present disclosure provide an imaging and measurement method, an apparatus, a device, and a storage medium for solving all or at least a part of the above problems.

[0004] A first aspect of the embodiment of the present disclosure is acquiring a plurality of sets of synchronized images of a measurement object having a surface provided with a plurality of marker points, the sets of synchronized images being continuously photographed by a compound eye camera, the plurality of sets of synchronized images each including a plurality of images photographed at the same time by a plurality of cameras in the compound eye camera; For each set of synchronized images, extracting coordinates of image points corresponding to the marker points in the synchronized images, and reconstructing first three-dimensional coordinates of the marker points corresponding to the image points based on the calibration data of the compound eye camera and the coordinates of the image points, thereby obtaining multiple sets of three-dimensional marker points; and constructing a global framework of marker points corresponding to the marker points on the surface of the object based on the sets of three-dimensional marker points.

[0005] A second aspect of the embodiment of the present disclosure is an acquisition module for acquiring a plurality of sets of synchronized images of a measurement object having a surface provided with a plurality of marker points, the sets of synchronized images being continuously photographed by a compound eye camera, the plurality of sets of synchronized images each including a plurality of images photographed at the same time by a plurality of cameras in the compound eye camera; A processing module is used to extract, for each set of synchronized images, coordinates of image points corresponding to the marker points in the synchronized images, and reconstruct first three-dimensional coordinates of the marker points corresponding to the image points according to the calibration data of the compound eye camera and the coordinates of the image points, thereby obtaining multiple sets of three-dimensional marker points; A construction module is used to construct a global framework of marker points corresponding to the marker points on the surface of the object based on the sets of three-dimensional marker points.

[0006] A third aspect of an embodiment of the present disclosure provides a photographing measuring device including a memory and a processor, the memory having a computer program stored therein, the computer program being capable of realizing the method of the first aspect when executed by the processor.

[0007] A fourth aspect of an embodiment of the present disclosure provides a computer-readable storage medium having a computer program stored thereon, the computer program being capable of implementing the method of the first aspect above when executed by a processor.

[0008] The technical solutions according to the embodiments of the present disclosure have the following advantages over the prior art:

[0009] In the embodiment of the present disclosure, a measurement object having a surface provided with a plurality of marker points is continuously photographed by a compound eye camera, and a plurality of sets of synchronized images each including a plurality of images photographed at the same time by a plurality of cameras in the compound eye camera are obtained, and the coordinates of the image points corresponding to the marker points in each set of synchronized images are extracted, and a first three-dimensional coordinate of the marker points corresponding to the image points is reconstructed based on the calibration data of the compound eye camera and the coordinates of the image points to obtain a plurality of sets of three-dimensional marker points, and a marker point global framework corresponding to the marker points on the surface of the measurement object is formed by the plurality of sets of three-dimensional marker points. The technical solution according to the embodiment of the present disclosure can realize matching of the same marker points when there are no or very few coding points by the compound eye measurement technology, thereby realizing shooting and measurement when there are no or very few coding points, reducing the installation work of the coding points by the measurer, improving measurement efficiency, and also realizing shooting and measurement of the measurement object even when there are no coding points, so that the measurement is not inaccurate due to the movement of the coding points, improving the accuracy of the measurement. [Brief description of the drawings]

[0010] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments contemplated by the present disclosure and, together with the description, serve to explain the principles of the present disclosure. In order to more clearly describe the technical solutions in the embodiments or prior art of the present disclosure, the drawings used in the description of the embodiments or prior art will be briefly described below. Needless to say, those skilled in the art can obtain other drawings based on these drawings without creative labor. R . [Figure 1] 1 is a flowchart of a photographing and measuring method according to an embodiment of the present disclosure. [Diagram 2] FIG. 2 illustrates a method for reconstructing 3D coordinates based on a polar matching method according to an embodiment of the present disclosure. [Diagram 3] 13 is a flowchart of another imaging and measuring method according to an embodiment of the present disclosure. [Figure 4]13 is a flowchart of yet another imaging and measuring method according to an embodiment of the present disclosure. [Diagram 5] 13 is a flowchart of yet another imaging and measuring method according to an embodiment of the present disclosure. [Figure 6] 13 is a flowchart of yet another imaging and measuring method according to an embodiment of the present disclosure. [Figure 7] FIG. 1 is a diagram showing a configuration of an imaging and measuring device according to an embodiment of the present disclosure. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0011] In order to make the above objectives, features and advantages of the present disclosure more clearly understood, the solutions of the present disclosure will be further described below. It should be noted that, unless contradictory, the embodiments and features in the embodiments of the present disclosure can be combined with each other.

[0012] In order to provide a thorough understanding of the present disclosure, numerous details are set forth in the following description; however, the present disclosure may be practiced in ways other than those described, and it is understood that the embodiments in the specification are merely some embodiments of the present disclosure, and not all embodiments.

[0013] 1 is a flow chart of a photographing and measuring method according to an embodiment of the present disclosure, and the method can be performed by a photographing and measuring device. The photographing and measuring device may be understood as a device having any image processing capability and computing capability. As shown in FIG. 1, the method according to the present embodiment includes steps 101 to 103.

[0014] In step 101, multiple sets of synchronized images of a measured object having multiple marker points on its surface are acquired, the multiple sets of synchronized images being continuously photographed by a compound eye camera, each set including multiple images photographed at the same time by multiple cameras in the compound eye camera.

[0015] The compound eye camera referred to in the embodiments of the present disclosure may be understood as a combination of cameras including two or more cameras. Before shooting, the compound eye camera can be first calibrated according to the calibration method according to the related art to obtain the internal parameters of each camera in the compound eye camera and the relative external parameters between each camera. For example, in a feasible calibration method, the compound eye camera can be calibrated according to the following method:

[0016] When the 3D coordinates and actual numbers of the marker points on the calibration plate are known, multiple sets of images are collected at different positions and different angles, and a bundle adjustment algorithm is used to minimize the coordinate error between the coordinates of the image points in the images of the marker points and the projected points in the images of the 3D coordinates of the marker points through iterative optimization, and then the internal parameters of each camera in the compound eye camera and the external parameters (i.e., external parameters) for the calibration plate at each shooting position of each camera are obtained, and the relative external parameters between each camera are determined based on the external parameters for the calibration plate at each shooting position of each camera, thereby completing the camera calibration.

[0017] The marker dots referred to in the embodiments of the present disclosure refer to marking figures (e.g., dots having a particular size, but not limited to dots) made of retroreflective material, and in the embodiments of the present disclosure, different marker dots may have different characteristics, such as, but not limited to, diameter and color.

[0018] Each set of synchronized images referred to in the embodiment of the present disclosure includes a plurality of images taken by a plurality of cameras in a compound eye camera at the same time, and may be understood as n images taken by n cameras at the same time by the same trigger signal, where each camera corresponds to one image, and n is a positive integer of 2 or more. Taking a binocular camera as an example, the binocular camera acquires a first set of synchronized images at a first time including a first image 11 acquired by a first camera at the first time and a second image 12 acquired by a second camera at the first time, and acquires a second set of synchronized images at a second time including a first image 21 acquired by a first camera at the second time and a second image 22 acquired by a second camera at the second time. Of course, the binocular camera is merely described as an example and is not intended to limit the embodiment of the present disclosure to one.

[0019] In one embodiment of the present disclosure, a plurality of marker points are provided on the surface of the object to be measured in advance, and the surface can be photographed by a compound-eye industrial camera, for example, a charge coupled device (CCD) camera. The compound-eye industrial camera is a key component in a machine vision system, and its most essential function is to convert an optical signal into a regular electrical signal. By moving the compound-eye industrial camera to continuously photograph the object to be measured, a plurality of sets of synchronized images of the object to be measured can be obtained.

[0020] In step 102, for each set of synchronized images, coordinates of image points corresponding to marker points in the synchronized images are extracted, and first 3D coordinates of the marker points corresponding to the image points are reconstructed based on the calibration data of the compound eye camera and the coordinates of the extracted image points, thereby obtaining multiple sets of 3D marker points.

[0021] Here, the "first 3D coordinate" referred to in the embodiments of the present disclosure is merely intended to distinguish between 3D coordinates obtained based on a 3D reconstruction method and 3D coordinates obtained based on other methods, and has no other meaning.

[0022] The first three-dimensional coordinates of the marker point may be understood as the coordinates of the marker point in a world coordinate system. Here, the world coordinate system may be understood as a coordinate system established in a three-dimensional space to describe the positional relationship between the camera and the object to be measured in the three-dimensional space, and the coordinate system is O w X w Y w Z w It can be shown that O w is the origin of the coordinate system, and X w is the x-axis component of the coordinate system, and Y w is the y-axis component of the coordinate system, and Z w is the z-axis component of the coordinate system. The origin of the world coordinate system can be set based on actual needs. (X w , Y w , Z w ) can be used to indicate the coordinates of the marker point in three-dimensional space.

[0023] A scene in space is projected by a pinhole model and imaged on a CCD, and then collected and stored as an image. In order to easily describe the image, it is necessary to define an image coordinate system, which is a two-dimensional coordinate system, the origin of which is set at the upper left corner of the image, and the x-axis and y-axis are flush with the synchronized image. The image point coordinates of a marker point in an image refer to the two-dimensional coordinates of the image point corresponding to the marker point in the image coordinate system. In the image coordinate system, the coordinates of the image point are in units of pixels, and each pixel can store the grayscale value of the image.

[0024] In photography measurement, it is further necessary to use the camera coordinate system and the image plane coordinate system. The camera coordinate system is mainly used for transformation from the world coordinate system to the image plane coordinate system, i.e., projection from 3D coordinates to a 2D image. The image plane coordinate system can record the 2D information onto which the marker points are projected, and complete the transformation from the 2D information to the coordinate system of the synchronized image.

[0025] Camera coordinate system O c X c Y c Z cThe origin is the optical center of the camera, and the optical axis of the camera is Z c axis of the camera coordinate system. c Axis and Y c The plane formed by the axes is parallel to the surface of the image plane (ie, the synchronous image).

[0026] The plane on which the coordinate system of the image plane exists is flush with the image plane, and the coordinate system of the image plane is defined as oxy. The origin o of the coordinate system of the image plane is the intersection of the optical axis and the image plane. The effective focal length f of the camera is defined as the distance from the optical center to the image plane, and the directions of the x-axis and y-axis of the coordinate system of the image plane respectively coincide with the pixel directions of the camera's image sensor.

[0027] In this embodiment, by extracting the coordinates in the synchronized image of the image point corresponding to the marker point in the synchronized image, the first three-dimensional coordinates in the world coordinate system of the marker point contained in the synchronized image can be reconstructed by a polar line matching method based on the previously obtained calibration data of the compound eye camera (including the internal parameters of each camera and the relative external parameters between each camera) and the coordinates in the synchronized image of the image point of the marker point. For example, FIG. 2 is a diagram showing a method for reconstructing three-dimensional coordinates based on the polar matching method. In FIG. 2, C1 and C2 may be understood as two cameras in a compound eye camera, the two cameras have undergone data calibration in advance, and the internal parameters of the two cameras and the relative external parameters between the two cameras have all been determined. O1 may be understood as the optical center of the C1 camera, O2 may be understood as the optical center of the C2 camera, and point P may be understood as any marker point on the surface of the object to be measured in space. When point P on the surface of the object to be measured is photographed simultaneously by the C1 and C2 cameras, it can be seen that the image point of point P in the synchronized image taken by the C1 camera is located at P1, and the two-dimensional coordinates of P1 in the synchronized image are extracted. All image points in the synchronized image of any point P' on the connecting line of O1P are P1. It can be seen that the three-dimensional coordinate of point P in the world coordinate system cannot be obtained simply by point P1 alone. Furthermore, it can be understood that the image point of point P in the synchronized image by the C2 camera is located at P2, the two-dimensional coordinates of P2 in the synchronized image are extracted, all image points in the synchronized image of any point on the connecting line of O2P are P2, the intersection of the two straight lines where O1P and O2P are located is the spatial position of the marker point P on the surface of the measured object, and the three-dimensional coordinates of point P are uniquely determined, and thus the three-dimensional coordinates of the marker point P on the surface of the measured object in the world coordinate system (i.e., the first three-dimensional coordinates referred to in the embodiment of the present disclosure) can be reconstructed. Of course, FIG. 2 here is merely an exemplary description of the method for reconstructing three-dimensional coordinates based on the polar matching method, and is not limiting.

[0028] In addition, in one embodiment of the present disclosure, when extracting the coordinates of the image points in the synchronized image of the marker point, an edge extraction method may be used. For example, Edge After performing the extraction process to obtain the image points in the synchronized image of the marker points, the coordinates of the image points in the coordinate system of the synchronized image may be determined based on the positions of the image points of the marker points in the synchronized image. Of course, the above method is only one method for extracting the coordinates of the image points of the marker points, and is not the only method.

[0029] The multiple sets of 3D marker points referred to in the embodiments of the present disclosure refer to 3D marker points obtained by processing based on each set of synchronized images in the multiple sets of synchronized images.

[0030] In an embodiment of the present disclosure, one corresponding 3D marker point can be obtained by calculation based on any image point in each set of synchronized images, and the 3D marker point calculated based on image points in different synchronized images may be the same.

[0031] In step 103, a global framework of marker points corresponding to the marker points on the surface of the object is constructed based on the sets of 3D marker points.

[0032] In an embodiment of the present disclosure, based on the acquired sets of 3D marker points, the sets of 3D marker points can be stitched together according to a certain order to construct a global framework of marker points corresponding to the marker points on the surface of the object to be measured.

[0033] In an embodiment of the present disclosure, a measured object having a plurality of marker points provided on its surface is photographed continuously by a compound eye camera, and multiple sets of synchronized images each including a plurality of images taken at the same time by multiple cameras in the compound eye camera are obtained, and the coordinates of the image points corresponding to the marker points in each set of synchronized images are extracted, and a first 3D coordinate of the marker point corresponding to the image point is reconstructed based on the calibration data of the compound eye camera and the coordinates of the image point to obtain multiple sets of 3D marker points, and a marker point global framework corresponding to the marker points on the surface of the measured object is constructed using the multiple sets of 3D marker points, and matching of the same marker points can be realized when there are no coding points or there are very few coding points, thereby realizing shooting and measurement when there are no coding points or there are very few coding points, reducing the work of installing coding points by the measurer and improving measurement efficiency. In addition, shooting and measurement of the measured object can be realized even when there are no coding points, so that the measurement is not inaccurate due to the movement of the coding points, and the accuracy of the measurement is improved.

[0034] In some embodiments of the present disclosure, a marker point global framework corresponding to the marker points on the surface of the object to be measured is constructed based on multiple sets of 3D marker points, and tracking and stitching processes and inter-set overlap removal processes are performed on the multiple sets of 3D marker points to obtain a marker point global framework corresponding to the marker points on the surface of the object to be measured.

[0035] In some embodiments, performing tracking and stitching processes and inter-set overlap removal processes for multiple sets of 3D marker points to obtain a global framework of marker points corresponding to the marker points on the surface of the object to be measured involves executing the flowchart of the photographing and measuring method shown in Figure 3, and as shown in Figure 3, the method according to this embodiment may include steps 301 to 305.

[0036] In step 301, multiple sets of synchronized images of a measured object having multiple marker points on its surface are acquired, the multiple sets of synchronized images being continuously photographed by a compound eye camera, each set including multiple images photographed at the same time by multiple cameras in the compound eye camera.

[0037] In step 302, for each set of synchronized images, the coordinates of the image points corresponding to the marker points in the synchronized images are extracted, and based on the calibration data of the compound eye camera and the coordinates of the image points, the first 3D coordinates of the marker points corresponding to the image points are reconstructed to obtain multiple sets of 3D marker points.

[0038] In step 303, a tracking and stitching process is performed on the multiple sets of 3D marker points to obtain original frameworks of marker points corresponding to the marker points on the surface of the object and numbers of each set of 3D marker points in the original framework.

[0039] In the embodiment of the present disclosure, after establishing a space triangle based on the 3D marker points in each pair of synchronized images, the triangles between the pairs are matched to each other, and the 3D marker points that form two congruent triangles are determined to be the same 3D marker points. The two synchronized images are stitched based on the correspondence between the congruent triangles between the pairs. In particular, if the correspondence between the pairs cannot be found, the correspondence between the pairs is searched for in the global framework. In this way, the tracking and stitching of multiple pairs of 3D marker points can be achieved to obtain the original framework of the marker points.

[0040] Here, when tracking the 3D marker points, due to the different characteristics of different 3D marker points, first extract the characteristics of each pair of 3D marker points, for example, a triangle formed between the 3D marker points, and then, based on the characteristics of each pair of 3D marker points, determine 3D marker points whose characteristic similarity is equal to or greater than a preset threshold as the same 3D marker point, and determine 3D marker points whose characteristic similarity is less than the preset threshold as different 3D marker points. In this way, the same 3D marker points in each pair are numbered with the same number and the different 3D marker points in each pair are numbered with different numbers, and the numbers of the 3D marker points in the original marker point framework of each pair of 3D marker points can be obtained. For example, if the first set of synchronized images are images taken consecutively by a camera the first time, and the second set of synchronized images are images taken consecutively by a camera the second time, there are five 3D marker points with different characteristics in the first set of synchronized images, which are numbered 1, 2, 3, 4, and 5, respectively, and there are six 3D marker points with different characteristics in the second set of synchronized images, of which five 3D marker points are the same as five 3D marker points in the first set of synchronized images, then these five 3D marker points in the second set of synchronized images will be numbered correspondingly as 1, 2, 3, 4, and 5, i.e., the same 3D marker points are numbered with the same numbers, and the remaining other 3D marker points will be numbered as 6, i.e., the different 3D marker points are numbered with different numbers.

[0041] In step 304, an inter-group overlap elimination process is performed on the numbers in the original framework of each set of 3D marker points to obtain a global framework of marker points corresponding to the marker points on the surface of the object to be measured and a unique number in the global framework of each set of 3D marker points.

[0042] In the embodiment of the present disclosure, each set of synchronized images is in a different coordinate system before stitching, and when stitching each set of synchronized images, each set of synchronized images needs to be transformed from the different coordinate systems in which they are located to the same coordinate system before stitching, and the transformation of different coordinate systems will generate system errors and cause errors to accumulate in the stitching process. Therefore, the embodiment of the present disclosure performs a process of removing overlaps between sets according to the number of each 3D marker point in the original framework, the 3D coordinates of each 3D marker point, and the position of the image point of each 3D marker point in each synchronized image, to obtain the unique number in the marker point global framework corresponding to the marker point on the surface of the object and the global framework of each set of 3D marker points, and ensure the uniqueness and consistency of the number in the global framework of each 3D marker point, thereby eliminating or reducing the accumulated error when transforming the coordinate system of each set of synchronized images during stitching, ensuring the accuracy of stitching multiple sets of synchronized images, and ensuring the uniqueness of the number in the global framework of each set of 3D marker points. Here, the inter-set overlap elimination process may be referred to as inter-frame overlap elimination process, and the method of performing the inter-set overlap elimination process in the embodiments of the present disclosure is similar to that of the related art, and detailed description thereof will be omitted here.

[0043] The technical solutions according to the embodiments of the present disclosure have the following advantages over the prior art:

[0044] In an embodiment of the present disclosure, a measured object having a plurality of marker points on its surface is successively photographed by a compound eye camera to obtain multiple sets of synchronized images, each including multiple images taken at the same time by multiple cameras in the compound eye camera, coordinates of image points corresponding to marker points in each set of synchronized images are extracted, and first 3D coordinates of the marker points corresponding to the image points are reconstructed based on calibration data of the compound eye camera and the coordinates of the image points to obtain multiple sets of 3D marker points, tracking and stitching processes are performed on the multiple sets of 3D marker points, an original framework of marker points corresponding to marker points on the surface of the measured object and numbers in the original framework of each set of 3D marker points are obtained, an inter-set overlap removal process is performed on the numbers in the original framework of each set of 3D marker points, and a unique number in the global framework of each set of 3D marker points is obtained. The technical solution according to the embodiment of the present disclosure combines compound eye measurement technology with tracking stitching technology and inter-group overlap removal technology to realize matching of the same marker points when there are no coding points or very few coding points, thereby realizing shooting measurement when there are no coding points or very few coding points, reducing the work of installing coding points by the measurer and improving measurement efficiency. In addition, since shooting measurement of the object to be measured can be realized even when there are no coding points, the measurement will not become inaccurate due to the movement of the coding points, and the measurement accuracy will be improved.

[0045] FIG. 4 is a flowchart of yet another photographing and measuring method according to an embodiment of the present disclosure. As shown in FIG.

[0046] In step 401, multiple sets of synchronized images of a measured object having multiple marker points on its surface are acquired, the multiple sets of synchronized images being continuously photographed by a compound eye camera, each set including multiple images photographed at the same time by multiple cameras in the compound eye camera.

[0047] In step 402, for each set of synchronized images, the coordinates of the image points corresponding to the marker points in the synchronized images are extracted, and based on the calibration data of the compound eye camera and the coordinates of the extracted image points, the first 3D coordinates of the marker points corresponding to the image points are reconstructed to obtain multiple sets of 3D marker points.

[0048] In step 403, a tracking and stitching process is performed on the multiple sets of 3D marker points, and an original framework of marker points corresponding to the marker points on the surface of the object to be measured and the number of each set of 3D marker points in the original framework are obtained.

[0049] In step 404, a process of removing overlaps between groups is performed on the numbers of the 3D marker points of each group in the original framework to obtain the global framework and a unique number of the 3D marker points of each group in the global framework.

[0050] In step 405, based on the coordinates of the image points in each set of synchronized images of the 3D marker points of each number in the global framework, the first 3D coordinates, and the internal parameters and external parameters of the compound eye camera, a bundle adjustment process is performed on the first 3D coordinates of the 3D marker points of each number to obtain the second 3D coordinates corresponding to the 3D marker points of each number.

[0051] Specifically, after completing the steps of reconstructing the first 3D coordinates of the above-mentioned marker points, tracking and stitching processing, and inter-group overlap removal processing, all 3D marker points in the obtained framework have unique numbers that have been determined, and there is a one-to-one corresponding relationship between the marker points included in each synchronized image and the 3D marker points in the global framework. In addition, by further adding the previously obtained internal parameters and external parameters of the compound eye camera, the coordinates of the image points in the synchronized images of each set of 3D marker points with each number, and the first 3D coordinates of the 3D marker points with each number, all the conditions required for bundle adjustment are met.

[0052] In some embodiments, the coordinates of the image points in each set of synchronized images of each numbered 3D marker point in the global framework, the first 3D coordinates, and the internal parameters and external parameters of the compound eye camera are used to calculate the coordinates of the image points in each set of synchronized images of each numbered 3D marker point. 1. Performing the bundle adjustment process on the three-dimensional coordinates and acquiring the second three-dimensional coordinates corresponding to the three-dimensional marker points of each number may include steps 40501 to 40502.

[0053] In step 40501, the interior and exterior orientation parameters of each set of synchronized images are input into the collinear equation respectively to obtain the collinear equation to be solved corresponding to each set of synchronized images.

[0054] In practice, the internal parameters of the camera include interior orientation parameters, distortion coefficients, vertical and horizontal pixel sizes, and vertical and horizontal pixel size ratios, and the interior orientation parameters include the horizontal translation distance x0 from the origin o of the coordinate system of the image plane (the intersection of the optical axis of the camera and the image plane) to the synchronized image center, the vertical translation distance y0 from the origin o of the coordinate system of the image plane to the synchronized image center, and the effective focal length f of the camera, which is the distance from the optical center of the camera to the image plane, that is, the interior orientation parameters indicate the spatial position of the optical center of the camera relative to the center of the synchronized image. The internal parameters of the camera may be obtained by camera calibration, and the interior orientation parameters may be considered to have already been determined in the above-mentioned camera calibration.

[0055] In the embodiment of the present disclosure, the parameters of the spatial position and attitude of the light beam at the moment of shooting can be determined by using the internal parameters and external parameters of the compound eye camera, and the parameters of the spatial position and attitude of the light beam at the moment of shooting are called exterior orientation elements, which are used to indicate the spatial position of the light beam at the moment of shooting. The exterior orientation elements include six parameters, three of which are line elements used to describe the spatial coordinate value of the shooting center, and the other three are angular elements used to describe the spatial attitude of the image.

[0056] Specifically, a collinear equation is a mathematical condition that states that three points, an object point, an image point, and the projection center (which for an image is usually the center of the lens), lie on a straight line.

[0057] In some embodiments of the present disclosure, the interior and exterior orientation parameters corresponding to each set of synchronized images can be input into the collinear equation to obtain the collinear equation to be solved corresponding to each set of synchronized images, and the collinear equation can be expressed by the following equation: JPEG0007675288000001.jpg30170Here, x, y are the coordinates of the image point of the marker point. x0, y0, f are the interior orientation parameters of the image. X s , Y s , Z s is the line element of the exterior orientation of the image. X A , Y A , Z A are the 3D coordinates of the marker points. a i , b i , c i (i=1, 2, 3) are the nine direction cosines composed of the angular elements of the three exterior orientations of the image.

[0058] In step 40502, based on the bundle adjustment algorithm, the coordinates of the image points in each set of synchronized images of each numbered 3D marker point in the global framework and the first 3D coordinates of each numbered 3D marker point are used as initial values, and iterative calculations are sequentially performed for all collinear equations to be solved to obtain the second 3D coordinates corresponding to each numbered 3D marker point.

[0059] Specifically, first, the coordinates of the image points in each set of synchronized images of the 3D marker points of each number in the global framework and the first 3D coordinates of the 3D marker points of each number are input as initial values ​​into the collinear equations to be solved corresponding to each set of synchronized images, and then iterative calculations are performed, and joint optimization is performed for the 3D marker points, the internal parameters of the compound eye camera, and the external parameters, to obtain the optimal coordinates of the 3D marker points when the residual error is smallest, that is, the second 3D coordinates of the 3D marker points. The process of the bundle adjustment algorithm according to the embodiment of the present disclosure is similar to that of the related art, and specifically, reference can be made to the bundle adjustment algorithm according to the related art, and detailed description thereof will be omitted here.

[0060] In this embodiment, by performing tracking and stitching processes on multiple sets of 3D marker points, the features of each set of 3D marker points are extracted to number the 3D marker points, and a process of removing overlaps between sets is performed on the numbers in the framework of each set of 3D marker points to ensure the uniqueness and consistency of the numbers in the framework of each set of 3D marker points. Based on the coordinates of the image points in the synchronized images of each set of 3D marker points of each number in the framework, the first 3D coordinates, and the internal parameters and external parameters of the compound eye camera, the 3D marker points of each number are calculated. 1. By performing bundle adjustment processing on the 3D coordinates and obtaining the second 3D coordinates corresponding to each numbered 3D marker point, i.e., the optimized 3D coordinates, the 3D images measured by the compound eye can be made more accurate, and the adverse effects of environmental factors such as temperature on the measurement results of the compound eye camera can be avoided. In addition, the reconstruction results can be checked in real time, the coordinates of the image points of the marker points can be directly processed, and the 3D coordinates of the marker points can be determined, which reduces the amount of calculation and improves the measurement efficiency.

[0061] In some embodiments of the present disclosure, a scale may be provided on the surface of the object to be measured or around the object to be measured, and multiple sets of synchronized images of the object to be measured taken successively by the compound eye camera include the object to be measured and the scale, i.e., the scale and the object to be measured are simultaneously photographed by the compound eye camera, and after obtaining unique numbers in the global framework of the 3D marker points in each set, the photographing measurement equipment may further execute the flowchart of the photographing measurement method according to Figure 5, and as shown in Figure 5, the method includes steps 501 to 503.

[0062] In step 501, a measurement dimension of at least one scale corresponding to an object to be measured and a physical dimension corresponding to the measurement dimension are obtained.

[0063] The scale in the embodiments of the present disclosure may be understood as a scale with known physical dimensions and may be used as a reference for photographic measurement, and the physical dimensions may be understood as actual dimensions, for example, the physical dimensions may include the actual length of the scale. Each scale may be composed of at least two coded marker points provided on the surface of the object to be measured or on the periphery of the object to be measured, and the coded marker points may be understood as special marker points with known coding information, and the physical dimensions of the scale may be obtained by the distance between the coded marker points. There is at least one scale corresponding to the object to be measured. In some embodiments, the scale may include a scale carrier, and the coded marker points may be provided on the scale carrier.

[0064] In an embodiment of the present disclosure, a measurement dimension of at least one scale corresponding to an object to be measured and a physical dimension corresponding to the measurement dimension can be obtained.

[0065] In some embodiments, the step of obtaining a measurement dimension of at least one scale corresponding to the object to be measured and a physical dimension corresponding to the measurement dimension may include steps 50101 to 50103.

[0066] In step 50101, based on the coding information of any two coded marker points on the scale, the second three-dimensional coordinates of the coded marker points corresponding to the coding information are obtained.

[0067] In an embodiment of the present disclosure, a first 3D coordinate of a marker point on the surface of the object to be measured and a first 3D coordinate of an encoded marker point at a scale corresponding to the object to be measured are reconstructed based on multiple sets of synchronized images, multiple sets of 3D marker points are obtained, and tracking and stitching processes, inter-set overlap removal processes, and bundle adjustment processes are performed on the multiple sets of 3D marker points to obtain second 3D coordinates of each 3D marker point, where the second 3D coordinates include the second 3D coordinates of the marker point on the surface of the object to be measured and the second 3D coordinates of the encoded marker point at a scale corresponding to the object to be measured, and based on the encoding information of any two encoded marker points at the scale, the second 3D coordinates of the encoded marker point corresponding to the encoding information are obtained.

[0068] In step 50102, based on the second three-dimensional coordinates of the coded marker points corresponding to the coded information, the distance between any two coded marker points is calculated to obtain a measurement dimension corresponding to the scale.

[0069] In an embodiment of the present disclosure, based on the second three-dimensional coordinates of the coded marker points corresponding to the coded information, the distance between any two coded marker points can be calculated and the distance can be determined as a measurement dimension corresponding to the scale.

[0070] In step 50103, a physical dimension corresponding to the measured dimension is determined based on the coded information of the coded marker point corresponding to the measured dimension.

[0071] In an embodiment of the present disclosure, the scale identification information and the physical dimension determined by any two coded marker points on the scale corresponding to the scale identification information can be pre-stored, and the scale identification information includes the coding information of each coded marker point on the scale.

[0072] In an embodiment of the present disclosure, based on the coding information of the coded marker point corresponding to the measurement dimension, identification information corresponding to the coding information can be determined, and the physical dimension corresponding to the measurement dimension can be obtained from the physical dimension of the scale corresponding to the identification information.

[0073] In step 502, the ratio of the measured dimension to the physical dimension of the scale is calculated.

[0074] In an embodiment of the present disclosure, after obtaining the measured dimensions and physical dimensions of the scales, the ratio between the measured dimensions and the physical dimensions of each scale can be calculated.

[0075] In step 503, adjust the second 3D coordinates of each 3D marker point in the global framework based on the ratio to obtain an adjusted global framework.

[0076] In an embodiment of the present disclosure, after obtaining a scale ratio corresponding to the object to be measured, one scale ratio can be selected, and the second three-dimensional coordinates of each three-dimensional marker point in the global framework are multiplied by the ratio to adjust the second three-dimensional coordinates of each three-dimensional marker point in the global framework, and an adjusted global framework is obtained.

[0077] In some embodiments, the step of adjusting the second three-dimensional coordinates of each three-dimensional marker point in the global framework based on the ratio and obtaining an adjusted global framework may include steps 50301 to 50302.

[0078] In step 50301, the average ratio of the scales corresponding to the object to be measured is calculated based on the number and ratio of the scales.

[0079] In an embodiment of the present disclosure, based on the number of scales and the ratio between the measurement dimension and the physical dimension of each scale, a quotient of the ratio and the number can be calculated to obtain an average ratio of the scales corresponding to the object to be measured.

[0080] In step 50302, adjust the second 3D coordinates of each 3D marker point in the global framework based on the average ratio to obtain an adjusted framework.

[0081] In an embodiment of the present disclosure, after obtaining an average ratio of the scale corresponding to the measured object, the second three-dimensional coordinates of each three-dimensional marker point in the global framework can be multiplied by the average ratio to adjust the second three-dimensional coordinates of each three-dimensional marker point in the global framework, thereby obtaining an adjusted global framework.

[0082] For example, three scales a, b, and c are provided on the surface of the object to be measured, the physical length of scale a is 100.1 mm, the physical length of scale b is 100.2 mm, and the physical length of scale c is 100.3 mm. For each scale, calculations are made based on the second three-dimensional coordinates corresponding to the coded marker points on the scales, so that the measured dimension of scale a is 99.9 mm, the measured dimension of scale b is 100.0 mm, and the measured dimension of scale c is 100.1 mm. The average ratio of the three scales is calculated as (100.1 / 99.9+100.2 / 100.0+100.3 / 100.1) / 3, and then the second three-dimensional coordinates of each three-dimensional marker point in the global framework are multiplied by the average ratio to adjust the second three-dimensional coordinates of the three-dimensional marker points in the global framework, and an adjusted global framework is obtained.

[0083] Thus, the second three-dimensional coordinates of each three-dimensional marker point in the global framework can be adjusted based on the scale corresponding to the object to be measured, the global framework can be optimized, and the accuracy of the photographing measurement can be improved.

[0084] FIG. 6 is a flowchart of yet another imaging and measuring method according to an embodiment of the present disclosure. As shown in FIG.

[0085] In step 601, multiple sets of synchronized images of a measured object having multiple marker points on its surface are acquired, the multiple sets of synchronized images being continuously photographed by a compound eye camera, each set including multiple images photographed at the same time by multiple cameras in the compound eye camera.

[0086] In step 602, for each set of synchronized images, the coordinates of the image points corresponding to the marker points in the synchronized images are extracted, and based on the calibration data of the compound eye camera and the coordinates of the image points, the first 3D coordinates of the marker points corresponding to the image points are reconstructed to obtain multiple sets of 3D marker points.

[0087] In step 603, a tracking and stitching process is performed on the multiple sets of 3D marker points, and an original framework of marker points corresponding to the marker points on the surface of the object to be measured and the number of each set of 3D marker points in the original framework are obtained.

[0088] In step 604, a process of removing overlaps between groups is performed on the numbers of the 3D marker points of each group in the original framework to obtain the global framework and a unique number of the 3D marker points of each group in the global framework.

[0089] In step 605, for any one of the monocular cameras in the compound eye cameras, an image corresponding to the monocular camera in each set of synchronized images is extracted.

[0090] In the embodiment of the present disclosure, the compound eye camera includes at least two monocular cameras. After obtaining the marker point global framework corresponding to the marker point on the surface of the object to be measured and the unique number in the global framework of the three-dimensional marker point in each set, for any monocular camera of the compound eye cameras, the image corresponding to the monocular camera in each set of synchronized images can be extracted.

[0091] In step 606, the coordinates of the image point in the image corresponding to the monocular camera of each numbered 3D marker point in the global framework are determined as first image point coordinates.

[0092] In an embodiment of the present disclosure, first, the coordinates of the image point in the image corresponding to the monocular camera of each numbered 3D marker point in the global framework are determined, and then the coordinates of the image point in the image corresponding to the monocular camera of each numbered 3D marker point in the global framework can be determined as the first image point coordinates.

[0093] In step 607, based on the first image point coordinates in the image corresponding to the monocular camera of each numbered 3D marker point in the global framework, the first 3D coordinates of each numbered 3D marker point, and the internal parameters and exterior orientation parameters of the monocular camera, a bundle adjustment process is performed on the first 3D coordinates of each numbered 3D marker point to obtain third 3D coordinates corresponding to each numbered 3D marker point.

[0094] In the embodiment of the present disclosure, the internal parameters of the monocular camera can be obtained by calibration, and the internal parameters of the monocular camera can be used to determine the spatial position and attitude parameters of the shooting light beam at the moment of shooting, and the spatial position and attitude parameters of the shooting light beam at the moment of shooting are called exterior orientation elements, which are used to indicate the spatial position of the shooting light beam at the moment of shooting. The exterior orientation elements include six parameters, three of which are line elements used to describe the spatial coordinate value of the shooting center, and the other three are angle elements used to describe the spatial attitude of the image.

[0095] In an embodiment of the present disclosure, based on the first image point coordinates in an image corresponding to a monocular camera of each numbered 3D marker point in a global framework, the first 3D coordinates of each numbered 3D marker point, and the internal parameters and exterior orientation parameters of the monocular camera, a bundle adjustment process is performed on the first 3D coordinates of each numbered 3D marker point to obtain the third 3D coordinates corresponding to each numbered 3D marker point.

[0096] Here, the "third 3D coordinates" referred to in the embodiments of the present disclosure are merely intended to distinguish 3D coordinates obtained based on the 3D reconstruction method from 3D coordinates obtained based on other methods, and have no other meaning.

[0097] In some embodiments, the step of performing a bundle adjustment process on the first 3D coordinates of each numbered 3D marker point based on the first image point coordinates in an image corresponding to the monocular camera of each numbered 3D marker point in the global framework, the first 3D coordinates of each numbered 3D marker point, and the internal parameters and exterior orientation parameters of the monocular camera to obtain the third 3D coordinates corresponding to each numbered 3D marker point may include steps 60701 to 60703.

[0098] In step 60701, a projective transformation is performed on the first three-dimensional coordinates of each numbered three-dimensional marker point to obtain the second image point coordinates of the image point in the image corresponding to the monocular camera of the first three-dimensional coordinates.

[0099] In an embodiment of the present disclosure, a projective transformation can be performed on the first three-dimensional coordinates of each numbered three-dimensional marker point to obtain second image point coordinates, which are the coordinates of the image point in an image corresponding to the monocular camera of the first three-dimensional coordinates.

[0100] In step 60702, a residual equation corresponding to each numbered 3D marker point is established based on the first image point coordinates and the second image point coordinates corresponding to each numbered 3D marker point.

[0101] In an embodiment of the present disclosure, after obtaining the first image point coordinates and the second image point coordinates corresponding to the first three-dimensional coordinates of each numbered three-dimensional marker point, a residual equation corresponding to each numbered three-dimensional marker point can be formulated based on the first image point coordinates and the second image point coordinates corresponding to each numbered three-dimensional marker point.

[0102] In step 60703, based on the bundle adjustment algorithm, the first 3D coordinates of each numbered 3D marker point in the global framework and the internal parameters and exterior orientation parameters of the monocular camera are set as initial values, and iterative calculations are performed sequentially for all residual equations to obtain the third 3D coordinates corresponding to each numbered 3D marker point.

[0103] In the embodiment of the present disclosure, the first 3D coordinates of the 3D marker points of each number in the global framework and the internal parameters and exterior orientation parameters of the monocular camera are used as initial values, and the iterative calculations for all residual equations corresponding to the 3D marker points of each number are sequentially performed to obtain the third 3D coordinates corresponding to the 3D marker points of each number. The process of the bundle adjustment algorithm according to the embodiment of the present disclosure is similar to that of the related art, and specifically, reference can be made to the bundle adjustment algorithm according to the related art, and detailed description thereof will be omitted here.

[0104] This makes it possible to perform a bundle adjustment process on the first 3D coordinates of the 3D marker points of each number based on the first image point coordinates in the image corresponding to the monocular camera of the 3D marker points of each number in the global framework, the first 3D coordinates of the 3D marker points of each number, and the internal parameters and exterior orientation elements of the monocular camera, and to obtain the third 3D coordinates corresponding to the 3D marker points of each number.This avoids the adverse effects on the measurement results of the compound eye cameras caused by structural instability factors that may occur between the compound eye cameras, and by obtaining optimized 3D coordinates, the 3D images measured by the compound eyes can be made more accurate.

[0105] 7 is a block diagram of a photographic measuring device according to an embodiment of the present disclosure, which may be understood as the photographic measuring device or a functional module of the photographic measuring device. As shown in FIG. 7, the photographic measuring device 700 includes: An acquisition module 701 is used for acquiring a plurality of sets of synchronized images of a measurement object having a surface provided with a plurality of marker points, the sets of synchronized images being continuously photographed by a compound eye camera, the plurality of sets of synchronized images each including a plurality of images photographed at the same time by a plurality of cameras in the compound eye camera; A processing module 702 is used for extracting, for each set of synchronized images, coordinates of image points corresponding to the marker points in the synchronized images, and reconstructing first three-dimensional coordinates of the marker points corresponding to the image points according to the calibration data of the compound eye camera and the coordinates of the image points, to obtain multiple sets of three-dimensional marker points; and a construction module 703 that is used to construct a global framework of marker points corresponding to the marker points on the surface of the object based on the sets of three-dimensional marker points.

[0106] Optionally, the configuration module 703:

[0107] and a processing sub-module used for performing tracking and stitching processes and overlap removal processes between the sets of 3D marker points to obtain a global framework of marker points corresponding to the marker points on the surface of the object to be measured.

[0108] Optionally, the processing submodule: A tracking and stitching unit is used for performing tracking and stitching processing on a plurality of sets of three-dimensional marker points, and obtaining marker point original frameworks corresponding to the marker points on the surface of the object to be measured, and numbers in the original framework of each set of three-dimensional marker points; and an inter-group overlap elimination unit, which is used to perform inter-group overlap elimination processing on the numbers of each group of three-dimensional marker points in the original framework, and obtain a global framework and a unique number in the global framework for each group of three-dimensional marker points.

[0109] Optionally, the processing module 702: For synchronous images Edgean extraction sub-module, which is used to perform an extraction process and obtain image points in the synchronized image; a determining sub-module that is used to determine coordinates in the coordinate system of the synchronized image of the image point based on the position of the image point in the synchronized image.

[0110] Optionally, the tracking and stitching unit comprises: an extraction subunit used for extracting features of each set of 3D marker points; a first determination subunit, which is used to determine, based on the features of each set of three-dimensional marker points, three-dimensional marker points whose feature similarity is equal to or greater than a preset threshold as the same three-dimensional marker point, and determine three-dimensional marker points whose feature similarity is less than the preset threshold as different three-dimensional marker points; and a numbering subunit that is used to number the same 3D marker points with the same number and to number different 3D marker points with different numbers.

[0111] Optionally, the processing submodule: The system further includes a first bundle adjustment unit, which is used to perform a bundle adjustment process on the first 3D coordinates of each numbered 3D marker point based on the coordinates of image points in each set of synchronized images of each numbered 3D marker point in the global framework, the first 3D coordinates, and the internal parameters and external parameters of the compound eye camera, to obtain second 3D coordinates corresponding to each numbered 3D marker point.

[0112] Optionally, the first bundle adjustment unit comprises: A first calculation subunit is used for inputting the interior orientation parameters and the exterior orientation parameters corresponding to each set of synchronized images into a collinear equation to obtain a collinear equation to be solved corresponding to each set of synchronized images; a second calculation subunit, which is used to, based on the bundle adjustment algorithm, take the coordinates of the image points in each set of synchronized images of each numbered 3D marker point in the global framework and the first 3D coordinates of each numbered 3D marker point as initial values, and sequentially perform iterative calculations for all collinear equations to be solved to obtain the second 3D coordinates of each numbered 3D marker point.

[0113] Optionally, the processing submodule: an acquisition unit used for acquiring a measurement dimension of at least one scale corresponding to the object to be measured and a physical dimension corresponding to the measurement dimension; a first calculation unit used to calculate a ratio between the measured dimension and the physical dimension; and an adjustment unit used for adjusting a second three-dimensional coordinate of each three-dimensional marker point in the framework based on the ratio to obtain an adjusted framework.

[0114] Optionally, the obtaining unit comprises: an acquisition subunit, which is used for acquiring, based on the coding information of any two coding marker points on the scale, a second three-dimensional coordinate of a coding marker point corresponding to the coding information; a third calculation subunit, which is used to calculate a distance between any two coded marker points according to the second three-dimensional coordinates of the coded marker points corresponding to the coded information, and obtain a measurement dimension corresponding to the scale; a second determining subunit used for determining a physical dimension corresponding to the measured dimension based on the coded information of the coded marker point corresponding to the measured dimension.

[0115] Optionally, the adjustment unit comprises: A fourth calculation subunit is used to calculate an average ratio of the scales corresponding to the object according to the number and ratio of the scales; and an adjustment subunit used for adjusting the second three-dimensional coordinates of each three-dimensional marker point in the global framework based on the average ratio to obtain an adjusted framework.

[0116] Optionally, the processing submodule: an extracting unit, which is used for extracting, for any one of the monocular cameras of the compound eye cameras, an image corresponding to the monocular camera in each set of synchronized images; A determination unit is used for determining the coordinates of an image point in an image corresponding to a monocular camera of each numbered 3D marker point in a global framework as a first image point coordinate; The system further includes a second bundle adjustment unit used to perform a bundle adjustment process on the first 3D coordinates of each numbered 3D marker point based on the first image point coordinates in an image corresponding to the monocular camera of each numbered 3D marker point in the global framework, the first 3D coordinates of each numbered 3D marker point, and the internal parameters and exterior orientation parameters of the monocular camera, to obtain third 3D coordinates corresponding to each numbered 3D marker point.

[0117] Optionally, the second bundle adjustment unit comprises: a projective transformation subunit, which is used to perform projective transformation of each numbered three-dimensional marker point to a first three-dimensional coordinate, and obtain a second image point coordinate of an image point in an image corresponding to the monocular camera of the first three-dimensional coordinate; an establishment subunit, which is used for establishing a residual equation corresponding to each number of three-dimensional marker points according to the first image point coordinates and the second image point coordinates corresponding to each number of three-dimensional marker points; and a fourth calculation subunit, which is used to obtain third 3D coordinates corresponding to each numbered 3D marker point in the global framework based on a bundle adjustment algorithm, and to sequentially perform iterative calculations for all residual equations using the first 3D coordinates of each numbered 3D marker point and the internal parameters and exterior orientation parameters of the monocular camera as initial values.

[0118] The photographing and measuring device according to the embodiments of the present disclosure can realize the method of any one of the above embodiments, and the implementation form and preferred effects of the method are similar, so detailed description will be omitted here.

[0119] An embodiment of the present disclosure provides a photographic measurement device, comprising: The apparatus includes a memory and a processor, the memory stores a computer program, and the computer program is executed by the processor to realize the above-mentioned photographing and measuring method. The implementation form and advantageous effects of the method are similar to those of the conventional method, and therefore a detailed description thereof will be omitted here.

[0120] An embodiment of the present disclosure provides a computer-readable storage medium, The storage medium stores a computer program, which, when executed by a processor, can realize the above-described photographing and measuring method. The implementation form and advantageous effects of the method are similar, and therefore a detailed description thereof will be omitted here.

[0121] The computer readable storage medium may be any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. The readable storage medium may include, but is not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples (non-exhaustive list) of readable storage media include an electrical connection having one or more conductors, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), optical fiber, a portable compact disk read-only memory (CD ROM), an optical storage device, a magnetic storage device, or any combination thereof.

[0122] The computer program may be written in any combination of one or more programming languages ​​to create program code for carrying out the operations of the embodiments of the present disclosure, including object-oriented programming languages ​​such as Java, C++, and the like, and further including general process programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server.

[0123] It should be noted that, in this specification, relational terms such as, for example, "first" and "second" are merely intended to distinguish one entity or operation from another and do not necessarily require or imply that any such actual relationship or sequence exists between those entities or operations. And, the terms "comprise", "have", or any other variations thereof are intended to cover a non-exclusive inclusion, such that a process, method, article, or device that includes a set of elements not only includes those elements, but also includes other elements not expressly listed, or includes elements inherent in such process, method, article, or device. Unless further limited, an element defined by the phrase "comprises a" does not exclude the presence of other identical elements in the process, method, article, or device that includes the element.

[0124] The above contents are merely embodiments of the present disclosure, which can be understood or realized by those skilled in the art. A number of modifications to these embodiments will be obvious to those skilled in the art, and the general principles defined herein can be realized in other embodiments without departing from the spirit or scope of the present disclosure. Therefore, the present disclosure is not limited to these embodiments described herein, but should be accorded the widest scope consistent with the principles and novel features disclosed herein. [Industrial Applicability]

[0125] The photography and measurement method disclosed herein uses compound eye measurement technology to enable matching of the same marker points when there are no coding points or very few coding points, thereby enabling photography and measurement when there are no coding points or very few coding points, reducing the work of installing coding points by the measurer and improving measurement efficiency.In addition, since photography and measurement of the object to be measured can be achieved even when there are no coding points, the measurement does not become inaccurate due to the coding points being moved, improving the accuracy of the measurement, and has extremely wide industrial applicability.

Claims

1. A photographic measurement method, comprising: acquiring a plurality of sets of synchronized images of a measurement object having a surface provided with a plurality of marker points, the sets of synchronized images being continuously photographed by a compound eye camera, the plurality of sets of synchronized images each including a plurality of images photographed at the same time by a plurality of cameras in the compound eye camera; For each set of synchronized images, extracting coordinates of image points corresponding to the marker points in the synchronized images, and reconstructing first three-dimensional coordinates of the marker points corresponding to the image points based on calibration data of the compound eye camera and the coordinates of the image points, thereby obtaining multiple sets of three-dimensional marker points; and acquiring a global framework of marker points corresponding to marker points on the surface of the object to be measured based on the plurality of sets of three-dimensional marker points.

2. The step of obtaining a global framework of marker points corresponding to marker points on a surface of the object based on the plurality of sets of three-dimensional marker points includes: The photographing and measuring method according to claim 1, further comprising a step of performing tracking and stitching processing and overlap removal processing between the sets of three-dimensional marker points to obtain a global framework of marker points corresponding to the marker points on the surface of the object to be measured.

3. The step of performing a tracking and stitching process and an overlap removal process on the plurality of sets of 3D marker points to obtain a global framework of marker points corresponding to the marker points on the surface of the object to be measured includes: performing a tracking and stitching process on the plurality of sets of three-dimensional marker points to obtain a marker point original framework corresponding to the marker points on the surface of the object to be measured and a number in the marker point original framework of each set of three-dimensional marker points; The photographing and measuring method according to claim 2, further comprising a step of performing an inter-group overlap elimination process on the numbers of each group of three-dimensional marker points in the marker point original framework to obtain the marker point global framework and unique numbers of each group of three-dimensional marker points in the marker point global framework.

4. The step of extracting coordinates of image points corresponding to the marker points in the synchronized images comprises: performing an edge extraction process on the synchronized image to obtain the image points in the synchronized image; 2. The method of claim 1, further comprising the step of: determining coordinates of the image point in the coordinate system of the synchronized image based on the position of the image point in the synchronized image.

5. The step of performing tracking and stitching processing on the sets of three-dimensional marker points includes: extracting features for each set of 3D marker points; determining, based on the features of each set of three-dimensional marker points, three-dimensional marker points whose feature similarity is equal to or greater than a preset threshold as the same three-dimensional marker point, and determining, based on the features of each set of three-dimensional marker points, three-dimensional marker points whose feature similarity is less than the preset threshold as different three-dimensional marker points; 4. The imaging and measuring method according to claim 3, further comprising the step of numbering the same three-dimensional marker points with the same number and numbering the different three-dimensional marker points with different numbers.

6. After the step of obtaining a unique number in the global framework of 3D marker points for each set of 3D marker points, The photographing and measuring method according to claim 3, further comprising a step of performing a bundle adjustment process on the first three-dimensional coordinates of each of the numbered three-dimensional marker points based on the coordinates of image points in each set of synchronized images of each numbered three-dimensional marker point in the marker point global framework, the first three-dimensional coordinates, and the internal parameters and external parameters of the compound eye camera, to obtain second three-dimensional coordinates corresponding to each of the numbered three-dimensional marker points.

7. the step of performing a bundle adjustment process on the first three-dimensional coordinates of each of the numbered three-dimensional marker points based on the coordinates of the image points in each set of synchronized images of the three-dimensional marker points of each number in the marker point global framework, the first three-dimensional coordinates, and the internal parameters and external parameters of the compound eye camera, to obtain second three-dimensional coordinates corresponding to the three-dimensional marker points of each number; Inputting the interior orientation parameters and exterior orientation parameters corresponding to each set of synchronized images into a collinear equation respectively to obtain a collinear equation to be solved corresponding to each set of synchronized images; The photographing and measuring method according to claim 6, further comprising a step of: based on a bundle adjustment algorithm, taking the coordinates of the image points in each set of synchronized images of the three-dimensional marker points of each number in the global framework of the marker points and the first three-dimensional coordinates of the three-dimensional marker points of each number as initial values, sequentially performing iterative calculations for all the collinear equations to be solved, and obtaining second three-dimensional coordinates corresponding to the three-dimensional marker points of each number.

8. After the step of acquiring second three-dimensional coordinates corresponding to the three-dimensional marker points of each number, obtaining a measurement dimension of at least one scale corresponding to the object to be measured and a physical dimension corresponding to the measurement dimension; calculating a ratio between the measured dimension and the physical dimension; The photographing and measuring method of claim 6, further comprising a step of adjusting second three-dimensional coordinates of each three-dimensional marker point in the marker point global framework based on the ratio to obtain an adjusted framework.

9. The step of obtaining a measurement dimension of at least one scale corresponding to the object to be measured and a physical dimension corresponding to the measurement dimension includes: obtaining, based on the coding information of any two coded marker points on the scale, second three-dimensional coordinates of the coded marker points corresponding to the coding information; calculating a distance between any two coded marker points based on second three-dimensional coordinates of the coded marker points corresponding to the coded information, and obtaining a measurement dimension corresponding to the scale; 9. The method of claim 8, further comprising the step of: determining a physical dimension corresponding to the measurement dimension based on coded information of a coded marker point corresponding to the measurement dimension.

10. The step of adjusting the second three-dimensional coordinates of each three-dimensional marker point in the global framework of marker points based on the ratio to obtain an adjusted framework includes: calculating an average ratio of the scales corresponding to the object based on the number of scales and the ratio; The photographing and measuring method of claim 8, further comprising a step of adjusting second three-dimensional coordinates of each three-dimensional marker point in the marker point global framework based on the average ratio to obtain an adjusted framework.

11. After the step of obtaining a unique number in the global framework of 3D marker points for each set of 3D marker points, extracting, for any one of the monocular cameras among the compound eye cameras, an image corresponding to the monocular camera in each set of synchronized images; determining image point coordinates in an image corresponding to the monocular camera of each numbered three-dimensional marker point in the marker point global framework as first image point coordinates; 4. The photographing and measuring method according to claim 3, further comprising a step of performing a bundle adjustment process on the first three-dimensional coordinates of each of the numbered three-dimensional marker points based on first image point coordinates in an image corresponding to the monocular camera of each of the numbered three-dimensional marker points in the marker point global framework, the first three-dimensional coordinates of each of the numbered three-dimensional marker points, and internal parameters and exterior orientation parameters of the monocular camera, to obtain third three-dimensional coordinates corresponding to each of the numbered three-dimensional marker points.

12. The step of performing a bundle adjustment process on the first three-dimensional coordinates of each numbered three-dimensional marker point based on first image point coordinates in an image corresponding to the monocular camera of each numbered three-dimensional marker point in the marker point global framework, the first three-dimensional coordinates of each numbered three-dimensional marker point, and internal parameters and exterior orientation parameters of the monocular camera to obtain third three-dimensional coordinates corresponding to each numbered three-dimensional marker point, performing a projective transformation of each numbered three-dimensional marker point with respect to a first three-dimensional coordinate, and acquiring a second image point coordinate of an image point in an image corresponding to the monocular camera of the first three-dimensional coordinate; formulating a residual equation corresponding to each number of the three-dimensional marker points based on the first image point coordinates and the second image point coordinates corresponding to each number of the three-dimensional marker points; The photographing and measuring method according to claim 11, further comprising a step of: setting the first three-dimensional coordinates of each numbered three-dimensional marker point in the marker point global framework and the internal parameters and exterior orientation parameters of the monocular camera as initial values ​​based on a bundle adjustment algorithm; sequentially performing iterative calculations for all the residual equations; and obtaining the third three-dimensional coordinates corresponding to each numbered three-dimensional marker point.

13. A photographic measuring device, comprising: an acquisition module for acquiring a plurality of sets of synchronized images of a measurement object having a surface provided with a plurality of marker points, the sets of synchronized images being continuously photographed by a compound eye camera, the plurality of sets of synchronized images each including a plurality of images photographed at the same time by a plurality of cameras in the compound eye camera; a processing module for extracting, for each set of synchronized images, coordinates of image points corresponding to the marker points in the synchronized images, and reconstructing first three-dimensional coordinates of the marker points corresponding to the image points based on calibration data of the compound eye camera and the coordinates of the image points, thereby obtaining a plurality of sets of three-dimensional marker points; and a construction module used to construct a global framework of marker points corresponding to marker points on the surface of the object based on a plurality of sets of three-dimensional marker points.

14. 1. A photographic measurement device including a memory and a processor, A photographing and measuring device, characterized in that a computer program is stored in the memory, and when the computer program is executed by the processor, the photographing and measuring method according to any one of claims 1 to 12 is realized.

15. A computer program product characterised in that, when executed by a processor, it implements the method according to any one of claims 1 to 12.

Images