Imaging data calibration method, electronic device, storage medium and program product

Abstract

The present invention discloses an imaging data calibration method, an electronic device, a storage medium and a program product. The method includes a processor and a memory, and a computer program cooperates to perform image correction under the execution of the processor. The correction process is: placing a preset calibration phantom at the center of the field of view of a CT imaging system and scanning it to generate two-dimensional projection image data; processing the image data and extracting projection feature points or areas related to the phantom; tracking the projection image locally frame by frame through the feature data to obtain trajectory data; based on this, performing projective geometry matching to achieve projective space reconstruction; converting the projective space reconstruction result to Euclidean space; using bundle adjustment to optimize the Euclidean space reconstruction result to obtain projection internal and external parameter matrices. This method does not require the management of a phantom geometry database, reduces management costs, avoids calibration errors caused by engineers' mistakes in selecting phantoms, and has a positive impact on process and risk prevention and control.

Description

Technical Field

[0001] The present invention relates to the field of image processing, and in particular to an imaging data calibration method, electronic equipment, storage medium and program product. Background Art

[0002] Cone Beam CT (CBCT) has important applications in the field of medical imaging, and its imaging quality directly affects the subsequent diagnosis and treatment effects. In order to ensure the accuracy of imaging, geometric calibration technology plays a vital role in CBCT systems. The purpose of CT geometric calibration is to accurately measure the relative geometric position relationship between the source, detector, and rotating gantry assembly. This geometric relationship will serve as an input parameter for the Filtered Back Projection (FBP) reconstruction algorithm, affecting the final quality of the image.

[0003] In a CT imaging system, the geometric relationship between the source and the detector can be equivalently represented by a pinhole imaging camera model, where the source is equivalent to the focus and the detector is equivalent to the image plane. The intrinsic geometric relationship between the source and the detector can be represented by a 3x3 matrix (intrinsic parameter matrix), and during the CT scanning process, the source and the detector rotate around the central axis of the scanned object, and their external geometric position changes can be described by a 3x4 matrix (extrinsic parameter matrix).

[0004] The traditional CT geometric calibration methods are mainly the following:

[0005] 1. Direct manual measurement method: geometric parameters are obtained by manual measurement. Although this method can be used during equipment installation and commissioning, its disadvantages are obvious, namely, the measurement process is time-consuming and labor-intensive, and it is difficult to ensure high accuracy and is easily affected by human errors.

[0006] 2. Known phantom measurement method: Use a calibration phantom with known geometric parameters for calibration. This method determines the geometric parameters of the phantom in advance, and uses the geometric features of the calibration phantom to match the projection feature data of the CT system from three-dimensional to two-dimensional (3D-2D matching) to achieve calibration. The commonly used phantom shape is cylindrical. However, this method has certain limitations. The accuracy of the phantom geometric parameters needs to be measured by other means, and the matching of features generally depends on specific encoding technology.

[0007] At present, with the continuous development of CT technology, research on geometric calibration is gradually moving towards high precision, automation and intelligence. How to complete the geometric calibration of the CT system more efficiently and accurately has become an urgent problem to be solved. Summary of the invention

[0008] The object of the present invention is to provide an imaging data calibration method, an electronic device, a storage medium and a program product to solve the problems raised in the above background technology.

[0009] To achieve the above object, the present invention provides the following technical solutions:

[0010] An imaging data calibration method, characterized in that it comprises: a processor and a memory, wherein a computer program is stored in the memory, and when the computer program is executed by the processor, the computer program and the processor cooperate to perform an image correction process, wherein the image correction process comprises the following steps:

[0011] Scanning calibration phantom: Place the preset calibration phantom in the center of the field of view of the CT imaging system, start the CT device to scan, and generate two-dimensional CT projection image data;

[0012] Image processing and feature extraction: Process the scanned projection image data to extract projection feature points or feature areas related to the surface shape, edge or surface structure of the phantom;

[0013] Local frame tracking: Using the extracted projection feature data, the local area of ​​the projection image is tracked frame by frame to obtain the projection feature trajectory data of each frame, reflecting the projection changes of the phantom at different projection angles;

[0014] Projective space reconstruction: Based on the extracted projection feature trajectory data, projective geometry matching of multi-frame projection features is performed to form a three-dimensional projective space reconstruction model;

[0015] Euclidean space conversion: convert the reconstruction result in the projective space into the Euclidean space to obtain a point cloud or geometric structure consistent with the three-dimensional Euclidean coordinate system;

[0016] Optimization of reconstruction results: The bundle adjustment technique is used to globally optimize the reconstruction results in the above Euclidean space to obtain the projection internal and external parameter matrices.

[0017] As a further solution of the present invention: the calibration mold body is a cylindrical mold body in which steel balls are embedded at fixed intervals.

[0018] As a further solution of the present invention: the feature matching method of the projection trajectory of the projection feature comprises the following steps:

[0019] The positions of each projected feature in the previous and next frames are connected in series to form the trajectory of the projected feature on the detector;

[0020] For any feature, the projection of the same feature in the current frame and the next frame is determined based on the fact that the projection position of the feature in the current frame and the next frame changes little and the displacement changes of the previous and next frames are consistent.

[0021] As a further solution of the present invention: based on the determined projection trajectory of the projection feature, feature matching is performed across multiple frames to find feature matching pairs between two frames.

[0022] As a further solution of the present invention: the projective space reconstruction process comprises the following steps:

[0023] Acquire feature matching relationships between local multi-frame images through motion tracking;

[0024] Using feature point matching and extreme geometric constraints, the basic matrix between frames is calculated;

[0025] Recover the relative depth of the projected object based on the fundamental matrix;

[0026] Homogenize the two-dimensional coordinates of the projected feature and then multiply them by their relative depth to obtain the corresponding three-dimensional coordinates;

[0027] Arrange the obtained three-dimensional coordinates into a matrix and repeatedly perform normalization operations on rows and columns;

[0028] Through singular value decomposition calculation, a series of 3×4 projection matrices and coordinates of four-dimensional space points are obtained.

[0029] As a further solution of the present invention: the Euclidean space conversion comprises the following steps:

[0030] A series of projection matrices in the four-dimensional projective space are obtained through projective space reconstruction;

[0031] Based on the invariance of the absolute quadratic curve in the projective space under any projection angle, a quadratic form is constructed, and the objective function is optimized by the nonlinear least squares method to solve the internal parameter matrix.

[0032] According to the solved intrinsic parameter matrix, a transformation from the projection space to the Euclidean space is performed, wherein the extrinsic parameter matrix is ​​obtained by reverse calculation of the intrinsic parameter matrix and the projection matrix, and then the three-dimensional feature points in the projection space are converted into three-dimensional feature points in the Euclidean space in combination with the extrinsic parameter matrix.

[0033] As a further solution of the present invention: the reconstruction result optimization process comprises the following steps:

[0034] The bundle adjustment method is used to perform overall constraint optimization on the internal parameter matrix, external parameter matrix and three-dimensional feature points.

[0035] By minimizing the optimization objective function of the projection error, the error between the projection feature point position and the projection coordinates of the three-dimensional feature point is minimized, where the projection error optimization objective function is: minimize the difference between the actual projection feature point position and the projection point position calculated by the intrinsic parameter matrix, the extrinsic parameter matrix and the three-dimensional feature point;

[0036] Decompose the external parameter matrix to obtain the rotation matrix and translation vector, where the translation vector represents the source position, and impose a coaxial circular motion constraint to ensure that the translation vector of each external parameter matrix is ​​on the same circle;

[0037] By minimizing the distance difference between the source position and the rotation center, the constraints of the translation vector are further optimized to reduce the deviation of the distance between the source position and the rotation center.

[0038] An electronic device, comprising:

[0039] a memory storing execution instructions; and

[0040] A processor is configured to execute the execution instructions stored in the memory.

[0041] A readable storage medium stores a computer program.

[0042] A computer program product comprising a computer program.

[0043] Compared with the prior art, the present invention has the following beneficial effects:

[0044] In some traditional methods, it is often necessary to measure the geometric data of the phantom itself in advance. However, this measurement process itself may introduce errors. For example, factors such as the accuracy limitation of the measuring tool and the instability of the measurement personnel's operation may lead to inaccurate geometric data of the measured phantom. Moreover, the phantom may be deformed during use due to various environmental factors or its own material properties. If the deformed geometric data is not updated in time, it will have a negative impact on the subsequent CT geometric calibration. This method does not require the pre-measurement of the phantom geometric data, and directly avoids these errors caused by the geometric errors of the phantom itself (whether caused by the measurement stage or the subsequent deformation) transmitted to the CT geometric calibration process, thereby improving the accuracy and reliability of CT calibration.

[0045] Traditional methods involve matching features from three dimensions to two dimensions, which is a relatively complex and difficult process. It requires finding the corresponding relationship between features in spatial data of different dimensions. Due to the span of dimensions, the structure of the data and the expression of the features have great changes, which makes the matching process prone to errors and inaccuracies.

[0046] This method replaces it with local frame 2D to 2D feature matching, that is, focusing on image feature matching. At the 2D image level, feature recognition and matching technology has developed quite maturely. For example, existing image feature extraction algorithms can efficiently extract feature information such as edges, corners, textures, etc. in images, and there are also many efficient methods for matching these 2D features that have been tested in practice. These algorithms are highly robust when processing image data, that is, they have strong resistance to interference factors such as noise, lighting changes, partial occlusion, etc. in the image, and can stably achieve feature matching in complex image environments, thereby ensuring the accuracy and stability of the entire process.

[0047] When it is necessary to manage the geometric database of the phantom itself, this requires a lot of human and material resources. On the one hand, a complete data storage system needs to be established to save the geometric data of different phantoms, including data entry, sorting, updating and other operations, which require dedicated personnel to be responsible. On the other hand, with the increase in the number of phantoms and changes during use, the maintenance cost of the database will continue to rise. Moreover, if the engineer makes a mistake when selecting the phantom for calibration, such as selecting a phantom that does not match the actual equipment or using defective phantom geometry data, it will lead to calibration errors, which will affect the performance of the entire CT system.

[0048] This method does not need to manage such a geometric database of the phantom itself, which directly reduces the management cost. There is no need to worry about database maintenance issues, and it also avoids the risk of calibration errors caused by engineers making mistakes in selecting from a large number of phantom data, which has a positive impact on both process simplification and risk prevention. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] Figure 1 Schematic diagram of the flow of imaging data calibration method M10.

[0050] Figure 2 This is a scanned projection of the phantom.

[0051] Figure 3 It is a schematic diagram of the process of projective space reconstruction S104 in the imaging data calibration method.

[0052] Figure 4 Schematic diagram of the process of Euclidean space transformation S105 in the imaging data calibration method.

[0053] Figure 5 A schematic diagram of the process of optimizing the reconstruction result S106 in the imaging data calibration method.

[0054] Figure 6 A schematic block diagram of the structure of an electronic device 1000 equipped with an imaging data calibration system according to an embodiment of the present invention. DETAILED DESCRIPTION

[0055] The present invention will be further described in detail below in conjunction with the accompanying drawings and implementations. It is to be understood that the specific implementations described herein are only used to explain the relevant content, rather than to limit the present invention. It is also necessary to explain that, for ease of description, only the parts related to the present invention are shown in the accompanying drawings.

[0056] It should be noted that, in the absence of conflict, the embodiments of the present invention and the features in the embodiments can be combined with each other. The technical solution of the present invention will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0057] Unless otherwise specified, the exemplary embodiments / embodiments shown will be understood as exemplary features that provide various details of some ways in which the technical concept of the present invention can be implemented in practice. Therefore, unless otherwise specified, the features of the various embodiments / embodiments can be combined, separated, interchanged and / or rearranged without departing from the technical concept of the present invention.

[0058] The terms used herein are for the purpose of describing specific embodiments, rather than being restrictive. As used herein, unless the context clearly indicates otherwise, the singular forms "one (kind, person)" and "said (the)" are also intended to include plural forms. In addition, when the terms "comprise" and / or "include" and their variations are used in this specification, it is explained that there are stated features, integral bodies, steps, operations, parts, assemblies and / or their groups, but it is not excluded that there are or add one or more other features, integral bodies, steps, operations, parts, assemblies and / or their groups. It should also be noted that, as used herein, the terms "substantially", "approximately" and other similar terms are used as approximate terms and not as degree terms, so that they are used to explain the inherent deviations of the measured values, calculated values ​​and / or provided values ​​that will be recognized by those of ordinary skill in the art.

[0059] Figure 1 A schematic diagram of the overall flow of an imaging data calibration method M10 according to some embodiments of the present invention is shown. Figure 1 The detailed description is for a clearer understanding of the technical solution of the present invention and should not be considered as a limitation of the protection scope of the present invention. In the process of implementing the technical idea of ​​the present invention, one or some steps may be omitted, or other alternatives may be adopted.

[0060] S101 Scanning a calibration phantom: firstly, placing a preset calibration phantom at the center of the field of view of the CT imaging system, and starting the CT device to scan the calibration phantom to generate two-dimensional CT projection image data;

[0061] S102 Image processing and feature extraction: Processing the projection image data obtained by scanning, extracting projection feature data of the phantom, wherein the projection feature includes feature points or feature areas corresponding to the surface shape, edge or surface structure of the phantom;

[0062] S103 local frame tracking: using the extracted projection feature data, tracking the local area of ​​the projection image frame by frame to obtain trajectory data of the projection feature of each frame, wherein the trajectory data reflects the projection change of the phantom at different projection angles;

[0063] S104 Projection space reconstruction: Based on the trajectory data of the projection features, feature matching is performed between local frames to form a projective geometric relationship between projection features of multiple frames, and three-dimensional reconstruction of the projective space is performed according to the projective geometric relationship to obtain a preliminary reconstruction result of the phantom;

[0064] S105 Euclidean space conversion: convert the reconstruction results in the projective space to the Euclidean space, and map the point cloud data or geometric structure in the projective space to the three-dimensional Euclidean coordinate system;

[0065] S106 Reconstruction result optimization: The reconstruction results in the above Euclidean space are globally optimized using the bundle adjustment technique to obtain the projection moments of the intrinsic parameters and the extrinsic parameters.

[0066] Before performing the CT scan calibration phantom operation, it is necessary to ensure that the calibration phantom is preset and meets relevant standards. It is crucial to place it accurately in the center of the field of view of the CT imaging system. The accuracy of this step will directly affect the quality and accuracy of subsequent image data. When the calibration phantom is properly placed, start the CT device to start the scanning process. During the scanning process, the CT device transmits and receives X-rays, and performs tomographic scanning on the calibration phantom according to the different absorption degrees of X-rays by human tissues or objects. Each scan will obtain information about the phantom from different angles. This information is processed and converted inside the device, and finally generates two-dimensional CT projection image data. These two-dimensional image data contain the projection information of the phantom at different angles, such as the grayscale value of different areas, which reflects the characteristics of the internal structure of the phantom at this projection angle.

[0067] There are many different design ideas and methods for phantom design. From a theoretical perspective, this solution is highly compatible and can adapt to a variety of phantom design methods. The key point is that no matter what design is adopted, it must ensure that the projection features of the phantom can be relatively easily identified and extracted.

[0068] In our current proposal, the phantom used is a cylinder with a specific structure. The design of this cylinder has steel balls embedded on its surface at regular intervals. These steel balls are evenly distributed on the cylinder, forming a unique structural pattern.

[0069] The reason why this cylinder with steel beads embedded in it is chosen as the phantom is that the steel beads can present very obvious and easily identifiable features during the projection process. Specifically, since the steel beads themselves are spherical in shape, when light is projected onto the phantom, the projection features presented by these steel beads on the projected image are still spherical in shape. This spherical projection feature has significant advantages in the subsequent image processing and feature extraction process, and it can be identified relatively simply and quickly.

[0070] From a visual point of view, Figure 2 As shown (assuming there is a corresponding phantom image here), you can clearly see the regularly arranged steel balls on the cylinder. The stability and recognizability of the projection features of these steel balls are very high under different lighting and projection angles, laying a good foundation for the subsequent operations of the entire imaging system, such as scanning, feature extraction, image reconstruction and other processes. Whether in the analysis of two-dimensional projection images or in the three-dimensional reconstruction process, these spherical projection features can serve as reliable marking points to help us more accurately restore the structure and shape information of the phantom.

[0071] Processing the two-dimensional projection image data obtained by scanning is a complex but critical step. First, the image needs to be preprocessed, such as removing noise, enhancing contrast, etc., to improve image quality and clarity. When extracting the projection feature data of the phantom, for features related to the shape of the phantom surface, it is necessary to analyze the change pattern of the grayscale value in the image to determine the boundary and contour feature points of the shape. If it is an edge feature, edge detection algorithms such as the Sobel operator and the Canny operator may be used. These algorithms detect the edge position by calculating the grayscale change gradient of the image pixel points in different directions, thereby determining the feature points or feature areas. For surface structural features, it may be necessary to identify areas with specific texture patterns as feature areas based on specific texture feature extraction algorithms. For example, for a phantom surface with a periodic structure, features can be extracted by analyzing periodic grayscale changes.

[0072] When using the extracted projection feature data for local frame tracking, we focus on the local area of ​​the projected image. In each frame, a local coordinate system is established based on the previously extracted feature points or feature areas. As the projection angle gradually changes, the position changes of these features in the local area are observed frame by frame. By recording the position movement of each feature point or feature area between different frames, their trajectory data at different projection angles can be obtained. For example, if a feature point is at coordinates (x1, y1) in the first frame, moves to (x2, y2) in the second frame, and so on, these continuous coordinate points constitute the trajectory of the feature point. These trajectory data can clearly reflect the projection changes of the model at different projection angles, such as how a local structure deforms or moves with the change of angle.

[0073] When reconstructing the projective space based on the trajectory data of the projection features, the first step is to perform feature matching between local frames. This means finding corresponding relationships between the projection features of different frames, such as matching a feature area of ​​a specific shape in one frame with an area of ​​a similar shape in another frame. Through this matching, the projective geometric relationship between the projection features of multiple frames can be determined. This geometric relationship describes how feature points and feature areas are related to each other under the projection transformation. Based on these projective geometric relationships, three-dimensional reconstruction of the projective space can be performed. In this process, mathematical models and algorithms are used to convert the feature information in the two-dimensional projection image into structural information in the three-dimensional space. For example, through feature matching and geometric calculations under multiple perspectives, the approximate shape and structure of the phantom in the three-dimensional projective space can be inferred, thereby obtaining the preliminary reconstruction results of the phantom.

[0074] Figure 3 The projective space reconstruction S104 in the imaging data calibration method of some embodiments of the present invention is shown. Step S104 may specifically include the following steps S1041 to S1045.

[0075] S1041 obtains the feature matching relationship between local multi-frame images through motion tracking: Motion tracking plays a key and fundamental role in the entire process of projective space reconstruction. By tracking the motion of the object, we can focus on the local multi-frame images and then explore the feature matching relationship between them. For example, we can deliberately select specific frames, such as [0,10,20] frames, which have a certain time interval. Between these frames, carefully analyze the position and morphological changes of the projection features of each steel ball. Starting from the first frame, mark the projection features of each steel ball, and then look for the corresponding features in the tenth and twentieth frames. This process requires careful comparison of every detail in the image, which may involve consideration of many factors such as the shape, size, and grayscale value of the features. In this way, we can establish a more accurate feature matching relationship between local multi-frames.

[0076] S1042 uses feature point matching and polar geometric constraints to calculate the basic matrix between frames: After the feature point matching is completed, it is necessary to use these matching information and pair-level geometric constraints to calculate the basic matrix between frames. In this process, the opencv library function findFundamentalMat can be used to complete the calculation. This function contains a complex algorithm, which performs operations based on the geometric relationship of feature point matching pairs. Specifically, it analyzes the positional relationship between corresponding feature points in different frames, and converts the position information on these two-dimensional planes into a basic matrix that can describe the geometric transformation between the two frames through a mathematical model. This basic matrix contains a comprehensive reflection of various geometric transformation information such as rotation, translation, and scaling between the two frames of images.

[0077] S1043 Restore the relative depth of the projected object according to the basic matrix: With the basic matrix F, we can further restore the relative depth of the steel ball projection. Calculate it using the following formula:

[0078]

[0079] Among them, eji represents the pole from the jth frame to the ith frame. It is a two-dimensional vector obtained by performing singular value decomposition on the basic matrix and is the smallest left singular vector. Singular value decomposition can be obtained through the svd function in the numpy library. This step is the key link in transforming two-dimensional image information into three-dimensional space information. The basic matrix F provides us with a bridge through which we can dig out the depth information hidden behind the image plane. The specific calculation process involves complex mathematical operations on the basic matrix and the coordinates of the image feature points. Through these operations, the relative depth position of each steel ball projection in the three-dimensional space can be derived. This relative depth information is not an absolute physical depth, but it provides an important basis for the subsequent construction of three-dimensional coordinates.

[0080] S1044 homogenizes the two-dimensional coordinates of the projected feature, multiplies them by their relative depth, obtains the corresponding three-dimensional coordinates, arranges the obtained three-dimensional coordinates into a matrix, and repeatedly performs normalization operations on rows and columns: First, the two-dimensional coordinates of the projected feature are homogenized. The homogenization process is to convert the two-dimensional coordinates into a form that can be easily combined with the depth information. Then, the homogenized two-dimensional coordinates are multiplied by the relative depth calculated previously, so that a series of three-dimensional coordinates can be obtained. These three-dimensional coordinates constitute a new matrix W. In the matrix W, each element It is a small 3*1 matrix, and the entire matrix W is 3m*n in size. Next, the normalization operation is repeated many times. Normalization is performed in the row direction and column direction respectively. In the row direction, each row is divided by the modulus of the row vector, so that the length of each row vector becomes 1, thereby eliminating the influence caused by the difference in vector length between different rows to a certain extent. In the column direction, each column is also divided by the modulus of the column vector. By repeatedly performing such normalization operations, the data in the matrix W can be made more stable and standardized.

[0081]

[0082] S1045 obtains a series of 3×4 projection matrices and coordinates of four-dimensional space points through singular value decomposition calculation: In the process of singular value decomposition, a series of 3x4 projection matrices P and four-dimensional space points X can be obtained. These projection matrices and space points are based on the representation of the projective space. In the projective space, although we have obtained these three-dimensional representations, it is different from the measurable Euclidean space we are familiar with. The points and geometric relationships in the projective space do not have the intuitive measurement properties such as distance and angle in the Euclidean space. It describes the spatial structure of objects more from the perspective of geometric transformation and projection relationship.

[0083] Converting the reconstruction results in the projective space to the Euclidean space is an important coordinate transformation process. In the projective space, the representation of points and geometric structures is different from that in the Euclidean space. The Euclidean space is the three-dimensional rectangular coordinate system that we are usually familiar with, with clear geometric measurement concepts such as distance and angle. In the conversion process, it is necessary to map the point cloud data or geometric structure in the projective space to the three-dimensional Euclidean coordinate system according to a specific mathematical transformation formula. This process involves complex coordinate transformations and geometric corrections to ensure that the reconstructed model structure in the converted Euclidean space conforms to our geometric cognition of the actual three-dimensional object. For example, for the coordinates (xp, yp, zp) of a point in the projective space, through a series of matrix operations and parameter adjustments, it is converted to the coordinates (xe, ye, ze) in the Euclidean space, so that subsequent analysis and processing can be more intuitive in the Euclidean space.

[0084] Figure 4 The Euclidean space transformation S105 in the imaging data calibration method of some embodiments of the present invention is shown. Step S105 may specifically include the following steps S1051 to S1053.

[0085] S1051 obtains a series of projection matrices in four-dimensional projective space through projective space reconstruction; in the process of transitioning from projective space to three-dimensional Euclidean space, the invariance of absolute quadratic curves is a key property. In projective geometry, absolute quadratic curves remain unchanged under any projection perspective. Based on this property, a specific quadratic form can be constructed. The construction of this quadratic form requires a deep understanding of the mathematical nature of projective space and absolute quadratic curves. It involves the construction of mathematical expressions for geometric elements in projective space, and the determination of the specific form of the quadratic form through complex mathematical derivations and operations so that it can accurately reflect the properties of absolute quadratic curves.

[0086] S1052 constructs a quadratic form based on the invariance of the absolute quadratic curve in the projective space under any projection angle, and optimizes the objective function through the nonlinear least squares method to solve the internal parameter matrix; then the nonlinear least squares method is used to optimize an objective function. The construction of this objective function is closely related to the quadratic form constructed previously and the geometric relationship in the projective space. During the optimization process, the nonlinear least squares algorithm will try to find a set of parameter values ​​that minimize the value of the objective function. This is an iterative process. In each iteration, the algorithm calculates the value of the objective function based on the current parameter value and adjusts the parameter value according to certain rules to gradually approach the optimal solution. This process requires an in-depth understanding of the mathematical properties of the objective function, and it is necessary to select a suitable optimization algorithm and initial parameter values ​​to ensure that the optimization process can quickly and stably converge to the optimal solution. Through the process of optimizing the objective function by nonlinear least squares, we can solve the internal parameter matrix. The internal parameter matrix contains some parameter information inside the camera, such as focal length, principal point coordinates, etc. These parameters reflect the internal geometric characteristics of camera imaging. The process of solving the internal parameter matrix involves extracting and analyzing the optimization results, screening out the parts related to the internal parameter matrix from the final parameter values ​​obtained by the optimization algorithm, and organizing them into the correct matrix form.

[0087] S1053 performs a transformation from the projection space to the Euclidean space according to the internal parameter matrix obtained by the solution, wherein the external parameter matrix is ​​obtained by reverse calculation of the internal parameter matrix and the projection matrix, and then the three-dimensional feature points in the projection space are converted into three-dimensional feature points in the Euclidean space in combination with the external parameter matrix. After obtaining the internal parameter matrix, the transformation matrix from the projection space to the Euclidean space can be determined. This transformation matrix is ​​the key bridge connecting the projection space and the Euclidean space. Its construction needs to comprehensively consider the internal parameter matrix and the geometric transformation relationship between the projection space and the Euclidean space. Then, the external parameter matrix and the three-dimensional feature points are solved by a specific formula. The external parameter matrix describes the position and posture information of the camera in the three-dimensional space, and it constitutes the complete parameters of the camera imaging model together with the internal parameter matrix. The process of solving the external parameter matrix and the three-dimensional feature points requires an in-depth understanding of the camera imaging model and related mathematical formulas, and the accuracy and stability of the calculation process must be guaranteed to obtain the correct external parameter matrix and three-dimensional feature points in the Euclidean space.

[0088] Figure 5 The reconstruction result optimization S106 in the imaging data calibration method of some embodiments of the present invention is shown. Step S106 may specifically include the following steps S1061 to S1064.

[0089] S1061 uses the bundle adjustment method to perform overall constraint optimization on the internal parameter matrix, external parameter matrix and three-dimensional feature points;

[0090] S1062 minimizes the error between the projection feature point position and the projection coordinates of the three-dimensional feature point by minimizing the optimization objective function of the projection error, wherein the projection error optimization objective function is: minimize the difference between the actual projection feature point position and the projection point position calculated by the internal parameter matrix, the external parameter matrix and the three-dimensional feature point. The optimization objective function of minimizing the projection error is the core of the entire optimization process. In actual operation, the projection feature point position is obtained by measuring the actual image, while the projection point position calculated by the internal parameter matrix, the external parameter matrix and the three-dimensional feature point is the predicted value based on the mathematical model. The essence of this objective function is to make the difference between the two as small as possible.

[0091] The intrinsic parameter matrix contains the imaging parameter information inside the camera, such as focal length, principal point coordinates, etc., which reflects the imaging characteristics of the camera itself. The extrinsic parameter matrix describes the position and posture of the camera in three-dimensional space, which is related to the external environment of the camera and the shooting scene. The three-dimensional feature point is the representation of the geometric features of the objects in the scene in three-dimensional space. When we calculate through these parameters, we are actually simulating the imaging process of the camera, projecting from points in three-dimensional space to the two-dimensional image plane. There is often a difference between the actual measured projection feature point position and the simulated calculated projection point position. This difference is the projection error that we want to minimize.

[0092] For a specific 3D feature point, its coordinates in 3D space are known. By performing projection calculations using the current intrinsic and extrinsic matrix, we can obtain a predicted projection point position on the image plane. At the same time, we also measure the actual projection position of the feature point in the actual image. By comparing the difference between these two positions and summing up all feature points, we can form the projection error optimization objective function.

[0093] S1063 decomposes the external parameter matrix to obtain the rotation matrix and translation vector, where the translation vector represents the source position, and applies a coaxial circular motion constraint to ensure that the translation vector of each external parameter matrix is ​​on the same circle. The purpose of decomposing the external parameter matrix is ​​to better understand and control the external posture of the camera. The external parameter matrix can be decomposed into a rotation matrix and a translation vector. Among them, the translation vector has a clear physical meaning, which represents the position of the source (camera). In some specific scenes, such as when the camera moves in a circle around a fixed point, a coaxial circular motion constraint can be applied. The purpose of this constraint is to ensure that the translation vector of each external parameter matrix is ​​on the same circle. In a geometric sense, this means that during the movement of the camera, its source position always remains on a circle with a certain center as the center. This is very important for some specific photogrammetry and computer vision applications because it can reflect the geometric structure of the shooting scene and the regularity of the camera movement.

[0094] S1064 further optimizes the constraints of the translation vector by minimizing the distance difference between the source position and the rotation center, reducing the deviation between the source position and the rotation center. By minimizing the distance difference between the source position and the rotation center, the constraints of the translation vector can be effectively optimized. For each translation vector of the external parameter matrix, we calculate its distance to the rotation center. Then, by comparing these distances with the mean of all distances, The differences between them are summed to form a new optimization objective function \min -mean By minimizing this objective function, all source positions can be made as close to an ideal circle as possible, thereby reducing the deviation between the source position and the rotation center and realizing the constraint of coaxial circular motion.

[0095] The purpose of using bundle adjustment technology to globally optimize the reconstruction results in Euclidean space is to improve the accuracy and precision of the reconstruction results. Bundle adjustment is an optimization algorithm based on the principle of least squares, which takes into account factors such as the propagation path of light during the imaging process and camera parameters. In this process, each point and geometric structure in the reconstruction result is regarded as formed by light projection, and the error between all projected light and the actual observed image features is minimized by adjusting the projection moments of intrinsic parameters (such as camera focal length, principal point coordinates, etc.) and extrinsic parameters (such as camera position and posture). For example, for the projection of a set of feature points, the theoretical projection position is calculated based on the current camera parameters and geometric model, and compared with the actual observed position. The adjustment amount of the parameters is calculated based on the error. Through multiple iterative optimizations, the error is gradually reduced, thereby obtaining more accurate projection moments of intrinsic and extrinsic parameters, making the reconstruction result closer to the real model structure.

[0096] The imaging data calibration method in the specific implementation manner of the present invention may be executed by an electronic device such as a mobile phone, a tablet computer, or a server.

[0097] Therefore, based on any of the above embodiments, the present invention also provides an electronic device, which can execute the imaging data calibration method of any of the embodiments described above in the present invention, and the imaging data calibration method of any of the embodiments described above can be configured on the electronic device.

[0098] Figure 61 is a block diagram of the structure of an electronic device 1000 configured with an imaging data calibration method according to an embodiment of the present invention. The hardware structure of the electronic device 1000 can be implemented using a bus architecture. The bus architecture can include any number of interconnecting buses and bridges, depending on the specific application and overall design constraints of the hardware. The bus 1100 connects various circuits including one or more processors 1200, memory 1300 and / or hardware modules together. The bus 1100 can also connect various other circuits 1400 such as peripherals, voltage regulators, power management circuits, external antennas, etc.

[0099] The bus 1100 may be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Component (EISA) bus, etc. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, the figure only uses one connecting line, but does not mean that there is only one bus or one type of bus.

[0100] The processor 1200 may be a central processing unit (CPU). The processor 1200 may also be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, or a combination of the above chips.

[0101] The memory 1300 can be used as a non-transitory computer-readable storage medium, and can be used to store non-transitory software programs, non-transitory computer executable programs and modules, such as program instructions of the computer program in the embodiment of the present invention. The processor 1200 implements the imaging data calibration method by running the non-transitory software programs, instructions and modules stored in the memory 1300.

[0102] The memory 1300 may include a program storage area and a data storage area, wherein the program storage area may store an operating system and an application required by at least one function; the data storage area may store data created by the processor 1200, such as structural data, model data, sample data, stiffness evaluation results, etc. of the bridge. In addition, the memory 1300 may include a high-speed random access memory, and may also include a non-transitory memory, such as at least one disk storage device, a flash memory device, or other non-transitory solid-state storage device. In some embodiments, the memory 1300 may optionally include a memory remotely arranged relative to the processor 1200, and these remote memories may be connected to the processor 1200 via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0103] The present invention also provides a readable storage medium, in which a computer program is stored, and when the computer program is executed by a processor, it is used to implement the above method. "Readable storage medium" can be any device that can contain, store, communicate, propagate or transmit a program for use in an instruction execution system, device or equipment or in combination with these instruction execution systems, devices or equipment. More specific examples of readable storage media include the following: an electrical connection portion (electronic device) with one or more wirings, a portable computer disk box (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable and editable read-only memory (EPROM or flash memory), an optical fiber device, and a portable read-only memory (CDROM), etc.

[0104] The present invention also provides a computer program product. The method of the present invention can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented by software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer programs or instructions. When the computer program or instruction is loaded and executed, the process or function of the present invention is executed in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, a network device, a user device, a core network device, an OAM or other programmable device.

[0105] The computer program or instructions may be stored in a readable storage medium or transmitted from one readable storage medium to another readable storage medium, for example, the computer program or instructions may be transmitted from one website, computer, server or data center to another website, computer, server or data center by wired or wireless means. The readable storage medium may be any available medium that can be accessed or a data storage device such as a server, data center, etc. that integrates one or more available media. The available medium may be a magnetic medium, such as a floppy disk, a hard disk, or a magnetic tape; it may also be an optical medium, such as a digital video disk; it may also be a semiconductor medium, such as a solid state drive. The computer readable storage medium may be a volatile or non-volatile storage medium, or may include both volatile and non-volatile types of storage media.

[0106] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0107] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0108] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0109] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0110] In the description of this specification, the description with reference to the terms "one embodiment / method", "some embodiments / methods", "example", "specific example", or "some examples" etc. means that the specific features, structures, or characteristics described in conjunction with the embodiment / method or example are included in at least one embodiment / method or example of the present invention. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment / method or example. Moreover, the specific features, structures, or characteristics described may be combined in any one or more embodiments / methods or examples in a suitable manner. In addition, those skilled in the art may combine and combine different embodiments / methods or examples described in this specification and features of different embodiments / methods or examples, unless they are contradictory.

[0111] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include at least one of the features. In the description of the present invention, the meaning of "plurality" is at least two, such as two, three, etc., unless otherwise clearly and specifically defined.

[0112] It should be understood by those skilled in the art that the above embodiments are only for the purpose of clearly illustrating the present invention, and are not intended to limit the scope of the present invention. For those skilled in the art, other changes or modifications may be made based on the above disclosure, and these changes or modifications are still within the scope of the present invention.

Claims

1. An imaging data calibration method, characterized in that: include: Scanning calibration phantom: Place the preset calibration phantom in the center of the field of view of the CT imaging system, start the CT device to scan, and generate two-dimensional CT projection image data; Image processing and feature extraction: Process the scanned projection image data to extract projection feature points or feature areas related to the surface shape, edge or surface structure of the phantom; Local frame tracking: Using the extracted projection feature data, the local area of ​​the projection image is tracked frame by frame to obtain the projection feature trajectory data of each frame, reflecting the projection changes of the phantom at different projection angles; Projective space reconstruction: Based on the extracted projection feature trajectory data, projective geometry matching of multi-frame projection features is performed to form a three-dimensional projective space reconstruction model; Euclidean space conversion: convert the reconstruction result in the projective space into the Euclidean space to obtain a point cloud or geometric structure consistent with the three-dimensional Euclidean coordinate system; Optimization of reconstruction results: The bundle adjustment technique is used to globally optimize the reconstruction results in the above Euclidean space to obtain the projection internal and external parameter matrices.

2. The imaging data calibration method according to claim 1, characterized in that: The calibration mold body is a cylindrical mold body inlaid with steel balls at fixed intervals.

3. The imaging data calibration method according to claim 1, characterized in that: The feature matching method of the projection trajectory of the projection feature comprises the following steps: The positions of each projected feature in the previous and next frames are connected in series to form the trajectory of the projected feature on the detector; For any feature, the projection of the same feature in the current frame and the next frame is determined based on the fact that the projection position of the feature in the current frame and the next frame changes little and the displacement changes of the previous and next frames are consistent.

4. The imaging data calibration method according to claim 1, characterized in that: The projective space reconstruction process comprises the following steps: Acquire feature matching relationships between local multi-frame images through motion tracking; Using feature point matching and extreme geometric constraints, the basic matrix between frames is calculated; Recover the relative depth of the projected object based on the fundamental matrix; Homogenize the two-dimensional coordinates of the projected feature and then multiply them by their relative depth to obtain the corresponding three-dimensional coordinates; Arrange the obtained three-dimensional coordinates into a matrix and repeatedly perform normalization operations on rows and columns; Through singular value decomposition calculation, a series of 3×4 projection matrices and coordinates of four-dimensional space points are obtained.

5. The imaging data calibration method according to claim 1, characterized in that: The Euclidean space conversion comprises the following steps: A series of projection matrices in the four-dimensional projective space are obtained through projective space reconstruction; Based on the invariance of the absolute quadratic curve in the projective space under any projection angle, a quadratic form is constructed, and the objective function is optimized by the nonlinear least squares method to solve the internal parameter matrix. According to the solved intrinsic parameter matrix, a transformation from the projection space to the Euclidean space is performed, wherein the extrinsic parameter matrix is ​​obtained by reverse calculation of the intrinsic parameter matrix and the projection matrix, and then the three-dimensional feature points in the projection space are converted into three-dimensional feature points in the Euclidean space in combination with the extrinsic parameter matrix.

6. The imaging data calibration method according to claim 1, characterized in that: The reconstruction result optimization process comprises the following steps: The bundle adjustment method is used to perform overall constraint optimization on the internal parameter matrix, external parameter matrix and three-dimensional feature points. By minimizing the optimization objective function of the projection error, the error between the projection feature point position and the projection coordinates of the three-dimensional feature point is minimized, where the projection error optimization objective function is: minimize the difference between the actual projection feature point position and the projection point position calculated by the intrinsic parameter matrix, the extrinsic parameter matrix and the three-dimensional feature point; Decompose the external parameter matrix to obtain the rotation matrix and translation vector, where the translation vector represents the source position, and impose a coaxial circular motion constraint to ensure that the translation vector of each external parameter matrix is ​​on the same circle; By minimizing the distance difference between the source position and the rotation center, the constraints of the translation vector are further optimized to reduce the deviation of the distance between the source position and the rotation center.

7. An electronic device, characterized in that: include: A memory storing execution instructions; as well as A processor, wherein the processor executes the execution instructions stored in the memory, so that the processor executes the imaging data calibration method according to any one of claims 1 to 6.

8. A readable storage medium, characterized in that: The readable storage medium stores a computer program, and when the computer program is executed by a processor, the computer program is used to implement the imaging data calibration method according to any one of claims 1 to 6.

9. A computer program product, characterized in that The computer program product comprises a computer program, and when the computer program is executed by a processor, the computer program is used to implement the imaging data calibration method according to any one of claims 1 to 6.

Images