A focal screen distance calibration method and device

Abstract

Embodiments of the present application provide a focal screen distance calibration method and device, and relate to the technical field of data processing. The method comprises: obtaining a perspective image of a registration component and a calibration component in a field of view range collected by a first image collection device; obtaining a first space coordinate of a first marker in a real three-dimensional space and a first image coordinate of the first marker in the perspective image; obtaining a coordinate conversion relationship between an image coordinate system corresponding to the perspective image and a space coordinate system corresponding to the real three-dimensional space based on the first space coordinate, the first image coordinate and a focal screen distance of the first image collection device; obtaining a second space coordinate of a second marker in a calibration component and a second image coordinate of the second marker; verifying whether the second space coordinate and the second image coordinate conform to the coordinate conversion relationship; and if not, calibrating the focal screen distance. The focal screen distance can be calibrated by the method provided by the embodiments of the present application, and the accuracy of the focal screen distance can be improved.

Description

Technical Field

[0001] This invention relates to the field of data processing technology, and in particular to a focal-to-screen distance calibration method and apparatus. Background Technology

[0002] With the gradual improvement of medical device research and development, more and more doctors are choosing to introduce surgical navigation and positioning systems during surgery to assist them in navigating and locating the lesion area of ​​the surgical patient, thereby completing the surgery and ensuring the surgical outcome.

[0003] In the process of navigating and locating the lesion area of ​​the surgical patient, the surgical navigation and positioning system needs to obtain fluoroscopic images acquired by image acquisition equipment such as X-ray machines and CT scanners. By analyzing the position of the lesion area in the fluoroscopic images, the spatial position of the lesion area in the real three-dimensional space is determined, thereby realizing navigation and positioning of the lesion area.

[0004] The above process requires the use of the focal length of the image acquisition device. In the prior art, the focal length of the image acquisition device is often provided by the device manufacturer. However, the focal length provided by the manufacturer generally has a large error compared to the actual focal length of the image acquisition device, which can easily lead to low accuracy in navigating and locating the lesion area by the surgical navigation and positioning system. Summary of the Invention

[0005] The purpose of this invention is to provide a focal-to-screen distance calibration method and apparatus to obtain a more accurate focal-to-screen distance. The specific technical solution is as follows:

[0006] In a first aspect, embodiments of the present invention provide a focal-to-screen distance calibration method, the method comprising:

[0007] Obtain perspective images of the registration and calibration components within the field of view acquired by the first image acquisition device;

[0008] Obtain the first spatial coordinates of the first marker in the real three-dimensional space and the first image coordinates of the first marker in the perspective image;

[0009] Based on the first spatial coordinates, the first image coordinates, and the focal-to-screen distance of the first image acquisition device, the coordinate transformation relationship between the image coordinate system corresponding to the perspective image and the spatial coordinate system corresponding to the real three-dimensional space is obtained.

[0010] Obtain the second spatial coordinates of the second marker in the real three-dimensional space and the second image coordinates of the second marker in the perspective image;

[0011] Verify whether the obtained second spatial coordinates and second image coordinates conform to the coordinate transformation relationship;

[0012] If not, then the focal distance is calibrated.

[0013] In one embodiment of the present invention, verifying whether the obtained second spatial coordinates and second image coordinates conform to the coordinate transformation relationship includes:

[0014] Based on the coordinate transformation relationship and the second spatial coordinates, calculate the projection coordinates of the second marker in the perspective image;

[0015] Verify whether the target distance between the projected coordinates and the second image coordinates is less than a preset distance. If it is less, determine that the second spatial coordinates and the second image coordinates conform to the coordinate transformation relationship. Otherwise, determine that the second spatial coordinates and the second image coordinates do not conform to the coordinate transformation relationship.

[0016] In one embodiment of the present invention, calibrating the focal distance includes:

[0017] Based on the target distance, determine the adjustment step size for the focal distance;

[0018] The focal distance is adjusted using the aforementioned adjustment step size;

[0019] The coordinate transformation relationship is updated based on the adjusted focal distance;

[0020] Update the projection coordinates of the second marker in the perspective image according to the updated coordinate transformation relationship;

[0021] Update the target distance between the projected coordinates and the second image coordinates based on the updated projected coordinates;

[0022] If the updated target distance is not less than the preset distance, return to the step of determining the adjustment step size for the focal distance based on the target distance, until the updated target distance is less than the preset distance.

[0023] In one embodiment of the present invention, the calibration component includes: a second marker, a positioning probe, and a first positioning marker, wherein the front end of the positioning probe is placed on the second marker, and the first positioning marker is installed on the tail end of the positioning probe;

[0024] Obtaining the second spatial coordinates of the second marker in the calibration component in real three-dimensional space includes:

[0025] Obtain the third spatial coordinates of the first positioning marker in the real three-dimensional space;

[0026] Based on the third spatial coordinates, the fourth spatial coordinates of the front end of the positioning probe in the real three-dimensional space are determined, and the fourth spatial coordinates are determined as the second spatial coordinates of the second marker in the calibration component in the real three-dimensional space.

[0027] In one embodiment of the present invention, obtaining the third spatial coordinates of the first positioning marker in real three-dimensional space includes:

[0028] Obtain a first target image containing the first positioning marker, captured by the second image acquisition device;

[0029] Identify the feature point coordinates of the first positioning marker in the first target image;

[0030] Based on the identified feature point coordinates, the first pose information of the first positioning marker relative to the second image acquisition device is obtained;

[0031] Based on the first pose information, the third spatial coordinates of the first positioning marker in the real three-dimensional space are determined.

[0032] In one embodiment of the present invention, the second marker is a marker ball;

[0033] The front end of the positioning probe is a hemispherical groove;

[0034] The inner diameter of the hemispherical groove is the same as the outer diameter of the marking ball.

[0035] In one embodiment of the present invention, the registration component includes: a first marker and a second positioning marker;

[0036] Obtaining the first spatial coordinates of the first marker in the registration component in real three-dimensional space includes:

[0037] Obtain the relative positional relationship between the second positioning marker and the first marker in the registration component;

[0038] Obtain the fifth spatial coordinates of the second positioning marker in real three-dimensional space;

[0039] Based on the fifth spatial coordinates and the relative positional relationship, the first spatial coordinates of the first marker in the registration component in the real three-dimensional space are determined.

[0040] In one embodiment of the present invention, obtaining the fifth spatial coordinates of the second positioning marker in real three-dimensional space includes:

[0041] Obtain a second target image containing the second positioning marker, captured by the second image acquisition device;

[0042] Identify the feature point coordinates of the second positioning marker in the second target image;

[0043] Based on the identified feature point coordinates, the second pose information of the second positioning marker relative to the second image acquisition device is obtained;

[0044] The fifth spatial coordinates of the second positioning marker are determined based on the second pose information.

[0045] Secondly, embodiments of the present invention provide a focal distance calibration device, the device comprising:

[0046] The perspective image acquisition module is used to obtain perspective images of the registration and calibration components within the field of view acquired by the first image acquisition device;

[0047] The first coordinate acquisition module is used to obtain the first spatial coordinates of the first marker in the real three-dimensional space and the first image coordinates of the first marker in the perspective image;

[0048] The coordinate transformation relationship acquisition module is used to obtain the coordinate transformation relationship between the image coordinate system corresponding to the perspective image and the spatial coordinate system corresponding to the real three-dimensional space based on the first spatial coordinates, the first image coordinates and the focal length of the first image acquisition device.

[0049] The second coordinate acquisition module is used to obtain the second spatial coordinates of the second marker in the real three-dimensional space and the second image coordinates of the second marker in the perspective image;

[0050] The coordinate verification module is used to verify whether the obtained second spatial coordinates and second image coordinates conform to the coordinate transformation relationship. If not, the focal length calibration module is triggered.

[0051] The focal-to-screen distance calibration module is used to calibrate the focal-to-screen distance.

[0052] In one embodiment of the present invention, the coordinate verification module includes:

[0053] The projection coordinate calculation submodule is used to calculate the projection coordinates of the second marker in the perspective image based on the coordinate transformation relationship and the second spatial coordinates;

[0054] The distance verification submodule is used to verify whether the target distance between the projected coordinates and the second image coordinates is less than a preset distance. If it is less, it is determined that the second spatial coordinates and the second image coordinates conform to the coordinate transformation relationship; otherwise, it is determined that the second spatial coordinates and the second image coordinates do not conform to the coordinate transformation relationship.

[0055] In one embodiment of the present invention, the focal distance calibration module includes:

[0056] The step size determination submodule is used to determine the adjustment step size for the focal-screen distance based on the target distance;

[0057] A focal-to-screen distance adjustment submodule is used to adjust the focal-to-screen distance using the adjustment step size;

[0058] The coordinate transformation relationship update submodule is used to update the coordinate transformation relationship based on the adjusted focal-to-screen distance.

[0059] The projection coordinate update submodule is used to update the projection coordinates of the second marker in the perspective image according to the updated coordinate transformation relationship;

[0060] The distance update submodule is used to update the target distance between the projected coordinates and the second image coordinates based on the updated projected coordinates; if the updated target distance is not less than the preset distance, the adjustment step size determination submodule is triggered until the updated target distance is less than the preset distance.

[0061] In one embodiment of the present invention, the calibration component includes: a second marker, a positioning probe, and a first positioning marker, wherein the front end of the positioning probe is placed on the second marker, and the first positioning marker is installed on the tail end of the positioning probe;

[0062] The second coordinate acquisition module includes:

[0063] The first spatial coordinate acquisition submodule is used to obtain the third spatial coordinates of the first positioning marker in real three-dimensional space;

[0064] The second spatial coordinate acquisition module is used to determine the fourth spatial coordinate of the front end of the positioning probe in the real three-dimensional space based on the third spatial coordinate, and to determine the fourth spatial coordinate as the second spatial coordinate of the second marker in the calibration component in the real three-dimensional space.

[0065] In one embodiment of the present invention, the first spatial coordinate acquisition submodule is specifically used to acquire a first target image containing the first positioning marker acquired by the second image acquisition device; identify the feature point coordinates of the first positioning marker in the first target image; obtain the first pose information of the first positioning marker relative to the second image acquisition device based on the identified feature point coordinates; and determine the third spatial coordinates of the first positioning marker in real three-dimensional space based on the first pose information.

[0066] In one embodiment of the present invention, the second marker is a marker ball;

[0067] The front end of the positioning probe is a hemispherical groove;

[0068] The inner diameter of the hemispherical groove is the same as the outer diameter of the marking ball.

[0069] In one embodiment of the present invention, the registration component includes: a first marker and a second positioning marker;

[0070] The first coordinate acquisition module includes:

[0071] A relative positional relationship acquisition submodule is used to obtain the relative positional relationship between the second positioning marker and the first marker in the registration component;

[0072] The third spatial coordinate submodule is used to obtain the fifth spatial coordinates of the second positioning marker in real three-dimensional space;

[0073] The fourth spatial coordinate acquisition submodule is used to determine the first spatial coordinates of the first marker in the registration component in the real three-dimensional space based on the fifth spatial coordinates and the relative position relationship.

[0074] In one embodiment of the present invention, the third spatial coordinate acquisition submodule is specifically used to acquire a second target image containing the second positioning marker acquired by the second image acquisition device; identify the feature point coordinates of the second positioning marker in the second target image; obtain the second pose information of the second positioning marker relative to the second image acquisition device based on the identified feature point coordinates; and determine the fifth spatial coordinates of the second positioning marker based on the second pose information.

[0075] Thirdly, embodiments of the present invention provide an electronic device, including a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus;

[0076] Memory, used to store computer programs;

[0077] When the processor executes the program stored in the memory, it implements the focal distance calibration method steps described in the first aspect above.

[0078] Fourthly, embodiments of the present invention provide a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the focal distance calibration method steps described in the first aspect.

[0079] Fifthly, embodiments of the present invention provide a computer program product containing instructions that, when run on a computer, cause the computer to perform the focus-screen distance calibration method steps described in the first aspect.

[0080] As can be seen from the above, when calibrating the focal length using the scheme provided in the embodiments of the present invention, the first image acquisition device first obtains the perspective images of the registration component and the calibration component within the field of view. Then, based on the first spatial coordinates of the first marker in the registration component in the real three-dimensional space, the first image coordinates in the perspective image, and the focal length of the first image acquisition device, the coordinate transformation relationship between the image coordinate system corresponding to the perspective image and the spatial coordinate system corresponding to the real three-dimensional space is obtained. Then, it is verified whether the second spatial coordinates of the second calibration object in the calibration component in the real three-dimensional space and the second image coordinates in the perspective image conform to the coordinate transformation relationship. If not, it indicates that there is an error between the focal length used when obtaining the coordinate transformation relationship and the actual focal length. Therefore, the focal length needs to be calibrated to reduce the error between the focal length and the actual focal length and obtain a more accurate focal length.

[0081] Of course, implementing any product or method of the present invention does not necessarily require achieving all of the advantages described above at the same time. Attached Figure Description

[0082] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other embodiments can be obtained based on these drawings.

[0083] Figure 1 A schematic flowchart of the first focal-to-screen distance calibration method provided in an embodiment of the present invention;

[0084] Figure 2 This is a schematic diagram illustrating a marker distribution method provided in an embodiment of the present invention;

[0085] Figure 3 A schematic diagram of a registration component provided in an embodiment of the present invention;

[0086] Figure 4 A schematic diagram of a positioning probe provided in an embodiment of the present invention;

[0087] Figure 5 This is a flowchart illustrating the second focal-to-screen distance calibration method provided in an embodiment of the present invention.

[0088] Figure 6 This is a flowchart illustrating the third focal-to-screen distance calibration method provided in an embodiment of the present invention.

[0089] Figure 7 This is a schematic diagram of the structure of the first focal-to-screen distance calibration device provided in an embodiment of the present invention;

[0090] Figure 8 This is a schematic diagram of the structure of the second focal-to-screen distance calibration device provided in an embodiment of the present invention;

[0091] Figure 9 This is a schematic diagram of the structure of the third focal-to-screen distance calibration device provided in an embodiment of the present invention;

[0092] Figure 10 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0093] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art based on this application are within the scope of protection of the present invention.

[0094] First, the subject responsible for implementing the solution provided in the embodiments of the present invention will be described.

[0095] The entity executing the solution provided in this embodiment of the invention can be any electronic device with data processing capabilities.

[0096] The focal distance calibration method provided in the embodiments of the present invention will be described in detail below.

[0097] See Figure 1 , Figure 1 This is a flowchart illustrating the first focal-to-screen distance calibration method provided in an embodiment of the present invention. The method includes the following steps S101-S106.

[0098] Step S101: Obtain perspective images of the registration and calibration components within the field of view acquired by the first image acquisition device.

[0099] The aforementioned fluoroscopic images can be understood as images that the aforementioned registration and calibration components can develop in the image, such as X-ray images, CT (Computed Tomography) images, and MR (Magnetic Resonance) images.

[0100] The first image acquisition device can be any type of device capable of acquiring the aforementioned fluoroscopic images, such as an X-ray machine, a CT (Computed Tomography) machine, or an MRI scanner. The aforementioned X-ray machine can be a C-arm X-ray machine, etc.

[0101] The registration component has multiple first markers, and the calibration component has multiple second markers. The registration component may contain multiple layers of first markers, and the calibration component may contain multiple layers of second markers.

[0102] Furthermore, the present invention does not limit the specific shape, material, or distribution of the first and second markers.

[0103] For example, the shape of the first marker and the second marker can be any shape that is easy to determine the geometric center, such as a sphere; the material of the first marker and the second marker can be any material that is easy for the first image acquisition device to image, such as steel; the distribution pattern of the first marker and the second marker can be any pattern that is easy to identify from the perspective image. The following uses the calibration component as an example to introduce a distribution pattern of the second marker in the calibration component.

[0104] See Figure 2 , Figure 2 This is a schematic diagram illustrating a marker distribution method provided by an embodiment of the present invention. The black dots in the diagram represent second markers, and it can be seen that the second markers can be distributed in a cross-shaped pattern as shown. This distribution method facilitates the clear display of the second markers in the perspective image and makes it easy to identify each second marker from the perspective image.

[0105] Specifically, the registration and calibration components can be positioned within the field of view of the first image acquisition device, enabling the first image acquisition device to capture the aforementioned perspective image. Furthermore, the registration and calibration components and the imaging plane of the first image acquisition device can be made parallel to each other to achieve better perspective image acquisition results.

[0106] Step S102: Obtain the first spatial coordinates of the first marker in the real three-dimensional space and the first image coordinates of the first marker in the perspective image in the registration component.

[0107] First, we will explain how to obtain the first image coordinates of the first marker in the perspective image.

[0108] Specifically, feature points of the first marker in the perspective image can be extracted, and the pixel coordinates of the feature points in the perspective image can be obtained as the aforementioned first image coordinates.

[0109] For example, if the first marker is spherical, the center of the first marker in the perspective image can be extracted as a feature point, and the pixel coordinates of the center in the perspective image can be obtained as the first image coordinates; or if the first marker is a cube, the geometric center of the first marker in the perspective image can be extracted as a feature point, and the pixel coordinates of the geometric center in the perspective image can be obtained as the first image coordinates.

[0110] The method for obtaining the first spatial coordinates of the first marker in real three-dimensional space will be explained below.

[0111] Specifically, the aforementioned first spatial coordinates can be obtained in the following ways.

[0112] In one implementation, the first spatial coordinates can be obtained based on a binocular camera. For example, an image containing a first marker is acquired using a binocular camera, the image coordinates of the first marker in the image are determined, and combined with pre-calibrated intrinsic and extrinsic parameters of the binocular camera and the distance between the binocular cameras, the first spatial coordinates of the first marker in the real three-dimensional space can be calculated according to the principle of binocular vision ranging. In this case, the real three-dimensional space is the real space where the binocular camera is located, and the first spatial coordinates are spatial coordinates in a coordinate system established based on the real space where the binocular camera is located.

[0113] In another implementation, the first spatial coordinates can be obtained based on a 3D depth sensor. For example, the distance between the first marker and the 3D depth sensor is measured using the 3D depth sensor. Using the pre-obtained spatial coordinates and pose information of the 3D depth sensor in the real three-dimensional space, the first spatial coordinates of the first marker in the real three-dimensional space can be calculated. In this case, the real three-dimensional space is the actual space where the 3D depth sensor is located, and the first spatial coordinates are spatial coordinates in a coordinate system established based on the actual space where the 3D depth sensor is located.

[0114] In another embodiment, the registration component may include a first marker and a second positioning marker. In this case, the spatial coordinates of the second positioning marker can be obtained, and the first spatial coordinates of the first marker can be indirectly obtained through the spatial coordinates of the second positioning marker. For details of the specific implementation, please refer to steps A-C in the following embodiments, which will not be described in detail here.

[0115] Of course, the above implementation is only an example, and any positioning technology that can be used for real three-dimensional spatial positioning can be used to obtain the above first spatial coordinates.

[0116] Step S103: Based on the first spatial coordinates, the first image coordinates, and the focal length of the first image acquisition device, obtain the coordinate transformation relationship between the image coordinate system corresponding to the perspective image and the spatial coordinate system corresponding to the real three-dimensional space.

[0117] The focal length of the first image acquisition device can be provided by the manufacturer of the first image acquisition device, or the focal length of the first image acquisition device can be calibrated using existing focal length calibration methods.

[0118] Specifically, after obtaining the first spatial coordinates, the first image coordinates, and the focal distance, the coordinate transformation relationship can be obtained using the following formula.

[0119]

[0120] In the above formula, s represents the pixel interval of the perspective image, (u, v) represents the first image coordinates of the first marker, and f x and f y Same, indicating focal length to screen distance, (c x c y (X, Y, Z) represents the vertical incidence center of the radiation source in the perspective image in the perspective image acquisition device, and (X, Y, Z) represents the first spatial coordinates of the first marker. The pixel interval and vertical incidence center are preset.

[0121] In the formula r 11 The matrix represented by -t3 can be called the registration matrix. It can be seen that all parameters outside the registration matrix in the formula are known. Substituting these known parameters into the formula yields the registration matrix. Then, substituting the registration matrix back into the formula as a known parameter allows us to calculate the first spatial coordinates of the first marker using its first image coordinates. This gives us the coordinate transformation relationship between the image coordinate system corresponding to the perspective image and the spatial coordinate system corresponding to the aforementioned real three-dimensional space.

[0122] Step S104: Obtain the second spatial coordinates of the second marker in the real three-dimensional space and the second image coordinates of the second marker in the perspective image.

[0123] First, we will explain how to obtain the second image coordinates of the second marker in the perspective image.

[0124] Specifically, feature points of the second marker in the perspective image can be identified, and the pixel coordinates of the feature points in the perspective image can be obtained as the aforementioned second image coordinates based on the extracted feature points.

[0125] The method for obtaining the second spatial coordinates of the second marker in real three-dimensional space will be explained below.

[0126] Specifically, the second spatial coordinates can be obtained by methods such as binocular camera positioning or 3D depth sensor positioning. For specific implementation methods, please refer to the first spatial coordinate acquisition method described in step S102 above, which will not be detailed here.

[0127] In one embodiment of the present invention, the calibration component may include: a second marker, a positioning probe, and a first positioning marker. The front end of the positioning probe is placed on the second marker, and the first positioning marker is installed on the tail end of the positioning probe. In this way, the coordinates of the probe tail can be calculated using the spatial coordinates of the first positioning marker, and then the coordinates of the probe tail can be used as the third spatial coordinates. For details of the specific implementation, please refer to steps D and E in the subsequent embodiments, which will not be described in detail here.

[0128] Step S105: Verify whether the obtained second spatial coordinates and second image coordinates conform to the coordinate transformation relationship. If not, proceed to step S106.

[0129] Since the coordinate transformation relationship is calculated based on the known focal length, if the above coordinate transformation relationship calculated based on the known focal length is accurate, the second spatial coordinates and the second image coordinates should satisfy the above coordinate transformation relationship.

[0130] Therefore, this step verifies whether the obtained second spatial coordinates and second image coordinates conform to the above coordinate transformation relationship to determine whether the known focal-to-screen distance is accurate: if yes, the determined coordinate transformation relationship is considered accurate, that is, the known focal-to-screen distance is accurate; if no, the coordinate transformation relationship is considered inaccurate, that is, the known focal-to-screen distance is inaccurate.

[0131] Specifically, the obtained second spatial coordinates and second image coordinates can be verified in the following ways to see if they conform to the above coordinate transformation relationship.

[0132] In one embodiment, the second spatial coordinates can be substituted into the coordinate transformation relationship formula calculated in step S103 to calculate the target image coordinates. If the target image coordinates are the same as the second image coordinates, it indicates that the second spatial coordinates and the second image coordinates conform to the above coordinate transformation relationship. If the target image coordinates are not the same as the second image coordinates, it indicates that the second spatial coordinates and the second image coordinates do not conform to the above coordinate transformation relationship.

[0133] In another implementation, the projected coordinates of the second marker in the perspective image can be calculated, and the target distance between the projected coordinates and the second image coordinates can be used to verify whether the second spatial coordinates and the second image coordinates conform to the coordinate transformation relationship. For detailed implementation methods, please refer to [link to implementation details]. Figure 5 Steps S506-S508 in the illustrated embodiment will not be described in detail here.

[0134] Step S106: Calibrate the above focal-to-screen distance.

[0135] The fact that the second spatial coordinates and the second image coordinates do not conform to the above coordinate transformation relationship indicates that there is an error between the known focal-to-screen distance and the actual focal-to-screen distance of the image acquisition device. In other words, the known focal-to-screen distance is inaccurate, and therefore the above focal-to-screen distance needs to be calibrated.

[0136] Specifically, the above focal length can be calibrated in the following ways.

[0137] In one embodiment, the focal length can be adjusted according to a preset adjustment step size. Based on the adjusted focal length, the formula introduced in step S103 is substituted to calculate the updated coordinate transformation relationship. The updated coordinate transformation relationship is then used to re-verify whether the second spatial coordinates and the second image coordinates conform to the updated coordinate transformation relationship. If not, the focal length adjustment step is repeated until the second spatial coordinates and the second image coordinates conform to the updated coordinate transformation relationship, thus completing the focal length calibration.

[0138] In another implementation, the projected coordinates of the second marker in the perspective image can be calculated, an adjustment step size can be determined based on the target distance between the projected coordinates and the second image coordinates, and the focal length can be adjusted according to the determined adjustment step size. For detailed implementation methods, see [link to implementation details]. Figure 6 Steps S609-S614 in the illustrated embodiment will not be described in detail here.

[0139] As can be seen from the above, when calibrating the focal length using the embodiments provided by the present invention, the first image acquisition device first obtains the perspective images of the registration component and the calibration component within the field of view. Then, based on the first spatial coordinates of the first marker in the registration component in the real three-dimensional space, the first image coordinates in the perspective image, and the focal length of the first image acquisition device, the coordinate transformation relationship between the image coordinate system corresponding to the perspective image and the spatial coordinate system corresponding to the real three-dimensional space is obtained. This verifies whether the second spatial coordinates of the second calibration object in the calibration component in the real three-dimensional space and the second image coordinates in the perspective image conform to the coordinate transformation relationship. If not, it indicates that there is an error between the focal length used to obtain the coordinate transformation relationship and the actual focal length. Therefore, the focal length with error can be calibrated to reduce the error between the focal length and the actual focal length and obtain a more accurate focal length.

[0140] In the first embodiment of the present invention, the calibrated focal-to-screen distance can be applied to the surgical navigation and positioning system.

[0141] In the process of navigating and locating the lesion area of ​​the surgical object, the surgical navigation and positioning system needs to analyze the fluoroscopic images acquired by the image acquisition device. The focal length of the image acquisition device is required in the above process, and the accuracy of the focal length affects the accuracy of the surgical navigation and positioning system in navigating and locating the lesion area. Therefore, by applying the solution provided in the embodiments of the present invention to obtain a more accurate focal length, the accuracy of the surgical navigation and positioning system in navigating and locating the lesion area can be improved.

[0142] In one embodiment of the present invention, the registration component may include: a first marker and a second positioning marker. In this case, the first spatial coordinates of the first marker can be obtained through the following steps A-C.

[0143] Step A: Obtain the relative positional relationship between the second positioning marker and the first marker in the registration component.

[0144] The aforementioned second positioning marker can be a marker capable of representing location information itself, or it can be a marker composed of multiple features, such as feature spheres, used to represent the location information of the second positioning marker. For example, the second positioning marker can be a tracer composed of multiple coplanar feature spheres, wherein the feature spheres can be arranged in a cross shape.

[0145] See Figure 3 , Figure 3 This is a schematic diagram of a registration component provided in an embodiment of the present invention. As can be seen from the figure, the first marker is a steel ball, and both the first marker and the second positioning marker are fixed on the registration component, and the relative positions of the first marker and the second positioning marker are fixed.

[0146] Specifically, by obtaining the spatial coordinates of the second positioning marker and the first marker in advance, and by obtaining the relative positional relationship based on the spatial coordinates, the first spatial coordinates of the first marker can be obtained during the execution of this scheme based solely on the spatial coordinates of the second positioning marker and the relative positional relationship, which is beneficial to improving the efficiency of obtaining the first spatial coordinates.

[0147] Step B: Obtain the fifth spatial coordinates of the second positioning marker in real three-dimensional space.

[0148] Specifically, the aforementioned fifth spatial coordinates can be obtained in the following ways.

[0149] In one embodiment, the second spatial coordinates can be obtained by means of binocular camera positioning, 3D depth sensor positioning, etc. For specific implementation methods, please refer to the first spatial coordinate acquisition method described in step S102 above. The only difference is that the positioning target is different, which will not be repeated here.

[0150] In another implementation, the aforementioned fifth spatial coordinates can be obtained through the following steps B1-B4.

[0151] Step B1: Obtain a second target image containing the second positioning marker, captured by the second image acquisition device.

[0152] The second target image can be a visible light image, and the aforementioned second image acquisition device can be any device capable of acquiring visible light images, such as a camera.

[0153] Step B2: Identify the feature point coordinates of the second positioning marker in the second target image.

[0154] The feature points of the second identifier can be feature points pre-set on the second identifier, or they can be characteristic points on the feature objects that make up the second identifier.

[0155] For example, if the second positioning marker is a tracer composed of four feature spheres, then the center coordinates of the four feature spheres can be identified from the second target image and used as the feature point coordinates of the second positioning marker. The method for identifying the feature point coordinates from the second target image can be referred to the aforementioned method. Figure 1 The method for identifying the first marker feature points from the perspective image as described in step S102 of the illustrated embodiment will not be repeated here.

[0156] Step B3: Based on the identified feature point coordinates, obtain the second pose information of the second positioning marker relative to the second image acquisition device.

[0157] Specifically, based on the relative positional relationship of each feature point in the predetermined second positioning marker, and combined with the coordinates of the identified feature points, the rotation angle and translation distance of the plane where the second positioning marker is located relative to the imaging plane of the second image acquisition device can be calculated, and the aforementioned rotation angle and translation distance can be used as the second pose information of the second positioning marker relative to the second image acquisition device.

[0158] Step B4: Determine the fifth spatial coordinates of the second positioning marker based on the second pose information.

[0159] Specifically, the spatial coordinates of the second image acquisition device in the real three-dimensional space can be determined in advance. As can be seen from step B3, the second pose information can characterize the rotation angle and translation distance between the plane where the second positioning marker is located and the imaging plane of the second image acquisition device. Therefore, the fifth spatial coordinates of the second positioning marker can be calculated using the spatial coordinates of the second image acquisition device and the second pose information.

[0160] This allows the fifth spatial coordinates of the second positioning marker to be obtained based on the second target image acquired by the second image acquisition device, without the need for a dedicated positioning device, thus reducing the cost required to obtain the fifth spatial coordinates of the second positioning marker.

[0161] Step C: Based on the fifth spatial coordinates and relative positional relationships, determine the first spatial coordinates of the first marker in the registration component in the real three-dimensional space.

[0162] After obtaining the fifth spatial coordinates of the second positioning marker, the first spatial coordinates can be calculated according to the relative positional relationship between the fifth spatial coordinates of the second positioning marker and the first spatial coordinates of the first marker.

[0163] In this way, by using only the fifth spatial coordinate of the second positioning marker, the first spatial coordinate of the first marker can be obtained according to the relative positional relationship between the second positioning marker and the first marker, which reduces the time required to obtain the first spatial coordinate of multiple first markers and improves the efficiency of obtaining the first spatial coordinate.

[0164] In one embodiment of the present invention, the calibration component may include: a second marker, a positioning probe, and a first positioning marker, wherein the front end of the positioning probe is placed on the second marker, and the first positioning marker is mounted on the tail end of the positioning probe. In this case, the second spatial coordinates of the second marker can be obtained through steps D and E.

[0165] Step D: Obtain the third spatial coordinates of the first positioning marker in the real three-dimensional space.

[0166] The first positioning marker mentioned above is a marker containing feature points, for example, it can be a tracer containing feature points.

[0167] See Figure 4 , Figure 4 This is a schematic diagram of a positioning probe provided in an embodiment of the present invention. As can be seen from the figure, a first positioning marker can be installed at the tail end of the positioning probe, and the tip end of the positioning probe can be placed on a second marker.

[0168] Specifically, the aforementioned third-space coordinates can be obtained in the following ways.

[0169] In one embodiment, the aforementioned third spatial coordinates can be obtained by means of binocular camera positioning, 3D depth sensor positioning, etc. For specific implementation methods, please refer to the first spatial coordinate acquisition method described in step S102 above. The only difference is that the positioning target is different, which will not be repeated here.

[0170] In another implementation, the aforementioned third spatial coordinates can be obtained through the following steps D1-D4.

[0171] Step D1: Obtain the first target image containing the first positioning marker, captured by the second image acquisition device.

[0172] Step D2: Identify the feature point coordinates of the first positioning marker in the first target image.

[0173] For example, if the first positioning marker is a tracer composed of four feature spheres, then the center coordinates of the four feature spheres can be identified from the first target image and used as the feature point coordinates of the second positioning marker. The method for identifying the feature point coordinates from the first target image can be referred to the aforementioned method. Figure 1 The method for identifying the first marker feature points from the perspective image as described in step S102 of the illustrated embodiment will not be repeated here.

[0174] Step D3: Based on the identified feature point coordinates, obtain the first pose information of the first positioning marker relative to the second image acquisition device.

[0175] This step can refer to the method described in step B3 above for obtaining the second pose information of the second positioning marker relative to the second image acquisition device. The only difference is that the feature points and positioning markers are different, which will not be repeated here.

[0176] Step D4: Based on the first pose information, determine the third spatial coordinates of the first positioning marker in the real three-dimensional space.

[0177] This step can refer to the method of obtaining the fifth spatial coordinates of the second positioning marker described in step B4 above. The only difference is that the positioning marker is different, so it will not be repeated here.

[0178] This allows the third spatial coordinates of the first positioning marker to be obtained based on the second target image acquired by the second image acquisition device, without the need for a dedicated positioning device, thus reducing the cost required to obtain the third spatial coordinates of the first positioning marker.

[0179] Step E: Based on the third spatial coordinates, determine the fourth spatial coordinates of the tail end of the positioning probe in the real three-dimensional space, and set the fourth spatial coordinates as the second spatial coordinates of the second marker in the calibration component in the real three-dimensional space.

[0180] Specifically, since the first positioning marker is installed at the tail end of the positioning probe, the third spatial coordinate of the first positioning marker is the spatial coordinate of the tail end of the positioning probe.

[0181] Therefore, based on the third spatial coordinates, the fourth spatial coordinates of the probe tip in real three-dimensional space can be calculated using the pre-measured probe length and the probe's axial direction in the first target image. Since the probe tip is placed on the second marker, the fourth spatial coordinates of the probe tip in real three-dimensional space can be determined as the second spatial coordinates of the second marker in real three-dimensional space.

[0182] By obtaining the fourth spatial coordinates of the positioning probe tip, the second spatial coordinates of the second marker can be determined conveniently and accurately.

[0183] In one embodiment of the present invention, the second marker may be a marker ball, the front end of the positioning probe is a hemispherical groove, and the inner diameter of the hemispherical groove is consistent with the outer diameter of the marker ball.

[0184] This allows the front end of the positioning probe to perfectly align with the marker ball, further improving the accuracy of obtaining the second spatial coordinates of the second marker from the third spatial coordinates of the first positioning marker.

[0185] In one embodiment of the present invention, the method of obtaining the second spatial coordinates of the second marker in the calibration component can also refer to the method of obtaining the first spatial coordinates of the first marker in the registration component described in steps A-C above. That is, the second spatial coordinates of each second marker are determined by only one positioning marker. This method is easy to obtain based on steps A-C, and will not be described again here.

[0186] exist Figure 1 Based on the illustrated embodiment, when verifying whether the obtained second spatial coordinates and second image coordinates conform to the coordinate transformation relationship, the projected coordinates of the second marker in the perspective image can be calculated. The conformity of the second spatial coordinates and second image coordinates with the coordinate transformation relationship is verified based on whether the target distance between the projected coordinates and the second image coordinates is greater than a preset distance. In view of the above, this embodiment of the invention provides a second focal-to-screen distance calibration method.

[0187] See Figure 5 , Figure 5 This is a flowchart illustrating the second focal distance calibration method provided in an embodiment of the present invention. The method includes the following steps S501-S508.

[0188] Step S501: Obtain perspective images of the registration and calibration components within the field of view acquired by the first image acquisition device.

[0189] Step S502: Obtain the first spatial coordinates of the first marker in the real three-dimensional space and the first image coordinates of the first marker in the perspective image in the registration component.

[0190] Step S503: Based on the first spatial coordinates, the first image coordinates, and the focal length of the first image acquisition device, obtain the coordinate transformation relationship between the image coordinate system corresponding to the perspective image and the spatial coordinate system corresponding to the real three-dimensional space.

[0191] Step S504: Obtain the second spatial coordinates of the second marker in the real three-dimensional space and the second image coordinates of the second marker in the perspective image.

[0192] Steps S501-S504 above are the same as those described above. Figure 1 In the illustrated embodiment, steps S101-S104 are the same and will not be repeated here.

[0193] Step S505: Based on the coordinate transformation relationship and the second spatial coordinates, calculate the projection coordinates of the second marker in the perspective image.

[0194] The above projection coordinates are the image coordinates of the projection of the second marker in the perspective image.

[0195] Specifically, the second spatial coordinates can be substituted into the coordinate transformation formula determined in step S103 above, and the calculated result is the projection coordinates of the second marker in the perspective image.

[0196] Step S506: Verify whether the target distance between the projected coordinates and the second image coordinates is less than the preset distance. If yes, proceed to step S507; if no, proceed to step S508.

[0197] Since the coordinate transformation relationship is calculated based on the known focal-to-screen distance, and the above-mentioned projected coordinates are the second spatial coordinates calculated based on the above-mentioned coordinate transformation relationship, if the above-mentioned coordinate transformation relationship calculated based on the known focal-to-screen distance is accurate, the projected coordinates should be the same as the actual second image coordinates of the second marker.

[0198] Therefore, the target distance between the projected coordinates and the second image coordinates reflects the magnitude of the error between the focal distance and the actual focal distance. The smaller the target distance, the more accurate the coordinate transformation relationship, which means the smaller the error between the focal distance and the actual focal distance. Conversely, the larger the target distance, the less accurate the coordinate transformation relationship, which means the larger the error between the focal distance and the actual focal distance.

[0199] The aforementioned preset distance reflects the acceptable focal length error. When the target distance is less than the preset distance, it indicates that the error between the focal length and the actual focal length is within an acceptable range, and the accuracy of the focal length is considered to meet the requirements. When the target distance is not less than the preset distance, it indicates that the error between the focal length and the actual focal length is not within an acceptable range, and the accuracy of the focal length is considered to not meet the requirements, requiring further calibration. The preset distance can be set by staff based on experience.

[0200] Step S507: Determine that the second spatial coordinates and the second image coordinates conform to the coordinate transformation relationship.

[0201] Step S508: Determine that the second spatial coordinates and the second image coordinates do not conform to the coordinate transformation relationship, and calibrate the focus screen distance.

[0202] The above step S508 is the same as the aforementioned Figure 1 Step S106 is the same in the embodiment, and will not be repeated here.

[0203] In this way, the target distance between the second image coordinates and the projection coordinates can intuitively reflect the error between the focal distance and the actual focal distance. Furthermore, when verifying whether the focal distance is accurate by checking whether the target distance is greater than the preset distance, since the preset distance reflects the acceptable focal distance error, different preset distances can be set according to different focal distance accuracy requirements, thereby obtaining a focal distance that meets the above accuracy requirements.

[0204] exist Figure 5 Based on the illustrated embodiment, the adjustment step size can be determined based on the target distance between the projected coordinates and the second image coordinates, and the focal-to-screen distance can be adjusted according to the determined adjustment step size. In view of the above, this embodiment of the invention provides a third method for focal-to-screen distance calibration.

[0205] See Figure 6 , Figure 6 This is a flowchart illustrating the third focal distance calibration method provided in an embodiment of the present invention. The method includes the following steps S601-S614.

[0206] Step S601: Obtain perspective images of the registration and calibration components within the field of view acquired by the first image acquisition device.

[0207] Step S602: Obtain the first spatial coordinates of the first marker in the real three-dimensional space and the first image coordinates of the first marker in the perspective image in the registration component.

[0208] Step S603: Based on the first spatial coordinates, the first image coordinates, and the focal length of the first image acquisition device, obtain the coordinate transformation relationship between the image coordinate system corresponding to the perspective image and the spatial coordinate system corresponding to the real three-dimensional space.

[0209] Step S604: Obtain the second spatial coordinates of the second marker in the real three-dimensional space and the second image coordinates of the second marker in the perspective image.

[0210] Step S605: Calculate the projection coordinates of the second marker in the perspective image based on the coordinate transformation relationship and the second spatial coordinates.

[0211] Steps S601-605 above are the same as those described above. Figure 5 Steps S501-S505 are the same in the embodiment, and will not be repeated here.

[0212] Step S606: Verify whether the target distance between the projected coordinates and the second image coordinates is less than the preset distance. If yes, proceed to step S607; if no, proceed to step S608.

[0213] Step S607: Determine that the second spatial coordinates and the second image coordinates conform to the coordinate transformation relationship.

[0214] Step S608: Determine that the second spatial coordinates and the second image coordinates do not conform to the coordinate transformation relationship.

[0215] Step S609: Based on the target distance, determine the adjustment step size for the focal distance.

[0216] Specifically, the adjustment step size can be determined according to the preset correspondence between the target distance and the adjustment step size, so that the adjustment step size can match the target distance. For example, the larger the target distance, the larger the adjustment step size is determined, and the smaller the target distance, the smaller the adjustment step size is determined.

[0217] Step S610: Adjust the focal distance using the step size adjustment method.

[0218] Step S611: Update the coordinate transformation relationship based on the adjusted focal distance.

[0219] Specifically, after adjusting the focal length, the registration matrix can be recalculated based on the adjusted focal length using the formula described in step S603 above. Substituting the recalculated registration matrix back into the formula yields the updated coordinate transformation relationship.

[0220] Step S612: Update the projection coordinates of the second marker in the perspective image according to the updated coordinate transformation relationship.

[0221] After the coordinate transformation relationship is updated, the updated projected coordinates can be calculated by substituting the second image coordinates of the second marker into the updated coordinate transformation relationship.

[0222] Step S613: Update the target distance between the projected coordinates and the second image coordinates based on the updated projected coordinates.

[0223] Corresponding to step S612, after the projection coordinates are updated, the target distance can be recalculated based on the updated projection coordinates to obtain the updated target distance.

[0224] Step S614: If the updated target distance is not less than the preset distance, return to step S609 until the updated target distance is less than the preset distance.

[0225] In this step, if the updated target distance is not less than the preset distance, it indicates that the accuracy of the adjusted focal distance has not yet reached the accuracy represented by the preset distance. Therefore, we can return to step S609 and continue to update and adjust the step size, coordinate transformation relationship, and projection coordinates according to the target distance, and then calculate the updated target distance until the updated target distance is less than the preset distance. At this point, it indicates that the accuracy of the focal distance has reached the requirement, and the focal distance calibration is complete.

[0226] This allows for continuous updating and adjustment of the step size based on the target distance after each update, ensuring that the adjustment step size matches the target distance and thus improving the calibration efficiency of the focal distance.

[0227] Of course, steps S607-S612 above are only one way to adjust the focal-to-screen distance based on the target distance. Alternatively, the target distance can be used as a loss value, and a convex optimization algorithm can be employed to adjust the focal-to-screen distance until the loss value is less than a preset distance, thus completing the focal-to-screen distance calibration. The convex optimization algorithm can be the least squares method, etc.

[0228] Corresponding to the above-described focal-to-screen distance calibration method, this embodiment of the invention also provides a focal-to-screen distance calibration device.

[0229] See Figure 7 , Figure 7 The first focal-to-screen distance calibration device provided in the embodiments of the present invention includes the following modules 701-706.

[0230] The perspective image acquisition module 701 is used to obtain perspective images of the registration and calibration components within the field of view acquired by the first image acquisition device;

[0231] The first coordinate acquisition module 702 is used to obtain the first spatial coordinates of the first marker in the real three-dimensional space and the first image coordinates of the first marker in the perspective image in the registration component;

[0232] The coordinate transformation relationship acquisition module 703 is used to obtain the coordinate transformation relationship between the image coordinate system corresponding to the perspective image and the spatial coordinate system corresponding to the real three-dimensional space based on the first spatial coordinates, the first image coordinates and the focal length of the first image acquisition device.

[0233] The second coordinate acquisition module 704 is used to obtain the second spatial coordinates of the second marker in the real three-dimensional space and the second image coordinates of the second marker in the perspective image;

[0234] The coordinate verification module 705 is used to verify whether the obtained second spatial coordinates and second image coordinates conform to the coordinate transformation relationship. If not, the focal length calibration module is triggered.

[0235] The focal-to-screen distance calibration module 706 is used to calibrate the focal-to-screen distance.

[0236] As can be seen from the above, when calibrating the focal length using the embodiments provided by the present invention, the first image acquisition device first obtains the perspective images of the registration component and the calibration component within the field of view. Then, based on the first spatial coordinates of the first marker in the registration component in the real three-dimensional space, the first image coordinates in the perspective image, and the focal length of the first image acquisition device, the coordinate transformation relationship between the image coordinate system corresponding to the perspective image and the spatial coordinate system corresponding to the real three-dimensional space is obtained. This verifies whether the second spatial coordinates of the second calibration object in the calibration component in the real three-dimensional space and the second image coordinates in the perspective image conform to the coordinate transformation relationship. If not, it indicates that there is an error between the focal length used to obtain the coordinate transformation relationship and the actual focal length. Therefore, the focal length with error can be calibrated to reduce the error between the focal length and the actual focal length and obtain a more accurate focal length.

[0237] In one embodiment of the present invention, the calibration component includes: a second marker, a positioning probe, and a first positioning marker, wherein the front end of the positioning probe is placed on the second marker, and the first positioning marker is installed on the tail end of the positioning probe;

[0238] The second coordinate acquisition module 704 includes:

[0239] The first spatial coordinate acquisition submodule is used to obtain the third spatial coordinates of the first positioning marker in real three-dimensional space;

[0240] The second spatial coordinate acquisition module is used to determine the fourth spatial coordinate of the front end of the positioning probe in the real three-dimensional space based on the third spatial coordinate, and to determine the fourth spatial coordinate as the second spatial coordinate of the second marker in the calibration component in the real three-dimensional space.

[0241] By obtaining the fourth spatial coordinates of the positioning probe tip, the second spatial coordinates of the second marker can be determined conveniently and accurately.

[0242] In one embodiment of the present invention, the first spatial coordinate acquisition submodule is specifically used to acquire a first target image containing the first positioning marker acquired by the second image acquisition device; identify the feature point coordinates of the first positioning marker in the first target image; obtain the first pose information of the first positioning marker relative to the second image acquisition device based on the identified feature point coordinates; and determine the third spatial coordinates of the first positioning marker in real three-dimensional space based on the first pose information.

[0243] This allows the third spatial coordinates of the first positioning marker to be obtained based on the second target image acquired by the second image acquisition device, without the need for a dedicated positioning device, thus reducing the cost required to obtain the third spatial coordinates of the first positioning marker.

[0244] In one embodiment of the present invention, the second marker is a marker ball;

[0245] The front end of the positioning probe is a hemispherical groove;

[0246] The inner diameter of the hemispherical groove is the same as the outer diameter of the marking ball.

[0247] This allows the front end of the positioning probe to perfectly align with the marker ball, further improving the accuracy of obtaining the second spatial coordinates of the second marker from the third spatial coordinates of the first positioning marker.

[0248] In one embodiment of the present invention, the registration component includes: a first marker and a second positioning marker;

[0249] The first coordinate acquisition module 702 includes:

[0250] A relative positional relationship acquisition submodule is used to obtain the relative positional relationship between the second positioning marker and the first marker in the registration component;

[0251] The third spatial coordinate submodule is used to obtain the fifth spatial coordinates of the second positioning marker in real three-dimensional space;

[0252] The fourth spatial coordinate acquisition submodule is used to determine the first spatial coordinates of the first marker in the registration component in the real three-dimensional space based on the fifth spatial coordinates and the relative position relationship.

[0253] In this way, by using only the fifth spatial coordinate of the second positioning marker, the first spatial coordinate of the first marker can be obtained according to the relative positional relationship between the second positioning marker and the first marker, which reduces the time required to obtain the first spatial coordinate of multiple first markers and improves the efficiency of obtaining the first spatial coordinate.

[0254] In one embodiment of the present invention, the third spatial coordinate acquisition submodule is specifically used to acquire a second target image containing the second positioning marker acquired by the second image acquisition device; identify the feature point coordinates of the second positioning marker in the second target image; obtain the second pose information of the second positioning marker relative to the second image acquisition device based on the identified feature point coordinates; and determine the fifth spatial coordinates of the second positioning marker based on the second pose information.

[0255] This allows the fifth spatial coordinates of the second positioning marker to be obtained based on the second target image acquired by the second image acquisition device, without the need for a dedicated positioning device, thus reducing the cost required to obtain the fifth spatial coordinates of the second positioning marker.

[0256] See Figure 8 , Figure 8 The second focal-to-screen distance calibration device provided in this embodiment of the invention includes the following modules 801-807.

[0257] The perspective image acquisition module 801 is used to obtain perspective images of the registration and calibration components within the field of view acquired by the first image acquisition device;

[0258] The first coordinate acquisition module 802 is used to obtain the first spatial coordinates of the first marker in the real three-dimensional space and the first image coordinates of the first marker in the perspective image in the registration component;

[0259] The coordinate transformation relationship acquisition module 803 is used to obtain the coordinate transformation relationship between the image coordinate system corresponding to the perspective image and the spatial coordinate system corresponding to the real three-dimensional space based on the first spatial coordinates, the first image coordinates and the focal length of the first image acquisition device.

[0260] The second coordinate acquisition module 804 is used to obtain the second spatial coordinates of the second marker in the real three-dimensional space and the second image coordinates of the second marker in the perspective image;

[0261] The projection coordinate calculation submodule 805 is used to calculate the projection coordinates of the second marker in the perspective image based on the coordinate transformation relationship and the second spatial coordinates;

[0262] The distance verification submodule 806 is used to verify whether the target distance between the projected coordinates and the second image coordinates is less than a preset distance. If it is less, it is determined that the second spatial coordinates and the second image coordinates conform to the coordinate transformation relationship; otherwise, it is determined that the second spatial coordinates and the second image coordinates do not conform to the coordinate transformation relationship.

[0263] The focal-to-screen distance calibration module 807 is used to calibrate the focal-to-screen distance if the condition is not met.

[0264] In this way, the target distance between the second image coordinates and the projection coordinates can intuitively reflect the error between the focal distance and the actual focal distance. Furthermore, when verifying whether the focal distance is accurate by checking whether the target distance is greater than the preset distance, since the preset distance reflects the acceptable focal distance error, different preset distances can be set according to different focal distance accuracy requirements, thereby obtaining a focal distance that meets the above accuracy requirements.

[0265] See Figure 9 , Figure 9 The third focal-to-screen distance calibration device provided in this embodiment of the invention includes the following modules 901-911.

[0266] The perspective image acquisition module 901 is used to obtain perspective images of the registration and calibration components within the field of view acquired by the first image acquisition device.

[0267] The first coordinate acquisition module 902 is used to obtain the first spatial coordinates of the first marker in the real three-dimensional space and the first image coordinates of the first marker in the perspective image in the registration component;

[0268] The coordinate transformation relationship acquisition module 903 is used to obtain the coordinate transformation relationship between the image coordinate system corresponding to the perspective image and the spatial coordinate system corresponding to the real three-dimensional space based on the first spatial coordinates, the first image coordinates and the focal length of the first image acquisition device.

[0269] The second coordinate acquisition module 904 is used to obtain the second spatial coordinates of the second marker in the real three-dimensional space and the second image coordinates of the second marker in the perspective image;

[0270] The projection coordinate calculation submodule 905 is used to calculate the projection coordinates of the second marker in the perspective image based on the coordinate transformation relationship and the second spatial coordinates;

[0271] The distance verification submodule 906 is used to verify whether the target distance between the projected coordinates and the second image coordinates is less than a preset distance. If it is less, it is determined that the second spatial coordinates and the second image coordinates conform to the coordinate transformation relationship; otherwise, it is determined that the second spatial coordinates and the second image coordinates do not conform to the coordinate transformation relationship.

[0272] The step size determination submodule 907 is used to determine the adjustment step size for the focal-screen distance based on the target distance;

[0273] The focal-to-screen distance adjustment submodule 908 is used to adjust the focal-to-screen distance using the adjustment step size;

[0274] The coordinate transformation relationship update submodule 909 is used to update the coordinate transformation relationship based on the adjusted focal-to-screen distance;

[0275] Projection coordinate update submodule 910 is used to update the projection coordinates of the second marker in the perspective image according to the updated coordinate transformation relationship;

[0276] The distance update submodule 911 is used to update the target distance between the projected coordinates and the second image coordinates according to the updated projected coordinates; if the updated target distance is not less than the preset distance, the adjustment step size determination submodule is triggered until the updated target distance is less than the preset distance.

[0277] This allows for continuous updating and adjustment of the step size based on the target distance after each update, ensuring that the adjustment step size matches the target distance and thus improving the calibration efficiency of the focal distance.

[0278] This invention also provides an electronic device, such as... Figure 10 As shown, it includes a processor 1001, a communication interface 1002, a memory 1003, and a communication bus 1004, wherein the processor 1001, the communication interface 1002, and the memory 1003 communicate with each other through the communication bus 1004.

[0279] Memory 1003 is used to store computer programs;

[0280] The processor 1001 is used to execute the program stored in the memory 1003 to implement the focal distance calibration method provided in the embodiments of the present invention.

[0281] The communication bus mentioned in the above electronic devices can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. This communication bus can be divided into address bus, data bus, control bus, etc. For ease of illustration, only one thick line is used to represent it in the diagram, but this does not mean that there is only one bus or one type of bus.

[0282] The communication interface is used for communication between the aforementioned electronic devices and other devices.

[0283] The memory may include random access memory (RAM) or non-volatile memory (NVM), such as at least one disk storage device. Optionally, the memory may also be at least one storage device located remotely from the aforementioned processor.

[0284] The processors mentioned above can be general-purpose processors, including central processing units (CPUs), network processors (NPs), etc.; they can also be digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0285] In another embodiment of the present invention, a computer-readable storage medium is also provided, which stores a computer program that, when executed by a processor, implements the focal distance calibration method provided in the embodiment of the present invention.

[0286] In another embodiment of the present invention, a computer program product containing instructions is also provided, which, when run on a computer, causes the computer to execute the focal distance calibration method provided in the embodiment of the present invention.

[0287] In the above embodiments, implementation can be achieved entirely or partially through software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented entirely or partially in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of the present invention are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid state disk (SSD)).

[0288] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0289] The various embodiments in this specification are described in a related manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on its differences from other embodiments. In particular, the embodiments for apparatus, electronic devices, and storage media are basically similar to the method embodiments, so the descriptions are relatively simple; relevant parts can be referred to the descriptions of the method embodiments.

[0290] The above description is merely a preferred embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention are included within the scope of protection of the present invention.

Claims

1. A focal screen distance calibration method, characterized by, The method comprises: obtaining a perspective image of a registration assembly and a calibration assembly in a field of view range collected by a first image collection device; obtaining a first space coordinate of a first marker in the registration assembly in a real three-dimensional space and a first image coordinate of the first marker in the perspective image; obtaining a coordinate conversion relationship between an image coordinate system corresponding to the perspective image and a space coordinate system corresponding to the real three-dimensional space based on the first space coordinate, the first image coordinate and a focal screen distance of the first image collection device; obtaining a second space coordinate of a second marker in the calibration assembly in the real three-dimensional space and a second image coordinate of the second marker in the perspective image; verifying whether the obtained second space coordinate and second image coordinate conform to the coordinate conversion relationship; if not, calibrating the focal screen distance.

2. The method of claim 1, wherein, The verification of whether the obtained second space coordinate and second image coordinate conform to the coordinate conversion relationship comprises: calculating a projection coordinate of the second marker in the perspective image based on the coordinate conversion relationship and the second space coordinate; verifying whether a target distance between the projection coordinate and the second image coordinate is less than a preset distance, if yes, determining that the second space coordinate and second image coordinate conform to the coordinate conversion relationship, otherwise, determining that the second space coordinate and second image coordinate do not conform to the coordinate conversion relationship.

3. The method of claim 2, wherein, The calibration of the focal screen distance comprises: determining an adjustment step for the focal screen distance based on the target distance; adjusting the focal screen distance by using the adjustment step; updating the coordinate conversion relationship based on the adjusted focal screen distance; updating the projection coordinate of the second marker in the perspective image according to the updated coordinate conversion relationship; updating the target distance between the projection coordinate and the second image coordinate according to the updated projection coordinate; if the updated target distance is not less than the preset distance, returning to the step of determining the adjustment step for the focal screen distance based on the target distance until the updated target distance is less than the preset distance.

4. The method of claim 1, wherein, The calibration assembly comprises a second marker, a positioning probe and a first positioning marker, a front end of the positioning probe is placed on the second marker, and the first positioning marker is installed at a tail end of the positioning probe. The obtaining of the second space coordinate of the second marker in the calibration assembly in the real three-dimensional space comprises: obtaining a third space coordinate of the first positioning marker in the real three-dimensional space; determining a fourth space coordinate of the front end of the positioning probe in the real three-dimensional space based on the third space coordinate, and determining the fourth space coordinate as the second space coordinate of the second marker in the real three-dimensional space.

5. The method of claim 4, wherein, The obtaining of the third space coordinate of the first positioning marker in the real three-dimensional space comprises: obtaining a first target image containing the first positioning marker collected by a second image collection device; identifying a feature point coordinate of the first positioning marker in the first target image; According to the identified feature point coordinates, first pose information of the first positioning marker relative to the second image acquisition device is obtained; Based on the first pose information, third space coordinates of the first positioning marker in real three-dimensional space are determined.

6. The method of claim 4 or 5, characterized in that, the second marker is a marker ball; the front end of the positioning probe is a semispherical groove; the inner diameter of the semispherical groove is consistent with the outer diameter of the marker ball.

7. The method according to any one of claims 1-5, characterized in that, The registration assembly includes a first marker and a second positioning marker; The first space coordinates of the first marker in real three-dimensional space in the registration assembly are obtained, including: obtaining the relative positional relationship between the second positioning marker and the first marker in the registration assembly; obtaining the fifth space coordinates of the second positioning marker in real three-dimensional space; based on the fifth space coordinates and the relative positional relationship, determining the first space coordinates of the first marker in real three-dimensional space in the registration assembly.

8. The method of claim 7, wherein, The fifth space coordinates of the second positioning marker in real three-dimensional space are obtained, including: obtaining a second target image containing the second positioning marker collected by a second image acquisition device; identifying feature point coordinates of the second positioning marker in the second target image; According to the identified feature point coordinates, second pose information of the second positioning marker relative to the second image acquisition device is obtained; based on the second pose information, the fifth space coordinates of the second positioning marker are determined.

9. A focal screen distance calibration device, characterized by The device includes: a perspective image acquisition module for obtaining a perspective image collected by a first image acquisition device on a registration assembly and a calibration assembly within a field of view range; a first coordinate obtaining module for obtaining first space coordinates of a first marker in real three-dimensional space in the registration assembly and first image coordinates of the first marker in the perspective image; a coordinate conversion relationship obtaining module for obtaining a coordinate conversion relationship between an image coordinate system corresponding to the perspective image and a space coordinate system corresponding to the real three-dimensional space based on the first space coordinates, the first image coordinates, and a focal screen distance of the first image acquisition device; a second coordinate obtaining module for obtaining second space coordinates of a second marker in real three-dimensional space in the calibration assembly and second image coordinates of the second marker in the perspective image; a coordinate verification module for verifying whether the obtained second space coordinates and second image coordinates conform to the coordinate conversion relationship, and triggering a focal screen distance calibration module if not; The focal screen distance calibration module is used to calibrate the focal screen distance.

10. The apparatus of claim 9, wherein, The coordinate verification module includes: a projection coordinate calculation submodule for calculating projection coordinates of the second marker in the perspective image based on the coordinate conversion relationship and the second space coordinates; The distance verification submodule is configured to verify whether a target distance between the projection coordinate and the second image coordinate is less than a preset distance. If the target distance is less than the preset distance, it is determined that the second spatial coordinate and the second image coordinate satisfy the coordinate conversion relationship. Otherwise, it is determined that the second spatial coordinate and the second image coordinate do not satisfy the coordinate conversion relationship.

11. The apparatus of claim 10, wherein, The focal screen distance calibration module comprises: The adjustment step determination submodule is configured to determine an adjustment step for the focal screen distance based on the target distance. The focal screen distance adjustment submodule is configured to adjust the focal screen distance by using the adjustment step. The coordinate conversion relationship updating submodule is configured to update the coordinate conversion relationship based on the adjusted focal screen distance. The projection coordinate updating submodule is configured to update the projection coordinate of the second marker in the perspective image according to the updated coordinate conversion relationship. The distance updating submodule is configured to update the target distance between the projection coordinate and the second image coordinate according to the updated projection coordinate. If the updated target distance is not less than the preset distance, the adjustment step determination submodule is triggered until the updated target distance is less than the preset distance.

12. The apparatus of claim 9, wherein, The calibration assembly comprises a second marker, a positioning probe, and a first positioning marker. The front end of the positioning probe is placed on the second marker, and the first positioning marker is installed at the tail end of the positioning probe. The second coordinate obtaining module comprises: The first spatial coordinate obtaining submodule is configured to obtain a third spatial coordinate of the first positioning marker in a real three-dimensional space. The second spatial coordinate obtaining submodule is configured to determine a fourth spatial coordinate of the front end of the positioning probe in a real three-dimensional space based on the third spatial coordinate, and determine the fourth spatial coordinate as a second spatial coordinate of the second marker in a real three-dimensional space in the calibration assembly.

13. The apparatus of claim 12, wherein The first spatial coordinate obtaining submodule is specifically configured to obtain a first target image containing the first positioning marker collected by a second image collection device, identify a feature point coordinate of the first positioning marker in the first target image, and obtain first pose information of the first positioning marker relative to the second image collection device according to the identified feature point coordinate. The fourth spatial coordinate of the front end of the positioning probe in a real three-dimensional space is determined based on the first pose information.

14. The apparatus of claim 12 or 13, wherein The second marker is a marker ball. The front end of the positioning probe is a semispherical groove. The inner diameter of the semispherical groove is consistent with the outer diameter of the marker ball.

15. The apparatus of any one of claims 9-13, wherein, The registration assembly comprises a first marker and a second positioning marker. The first coordinate obtaining module comprises: The relative position relationship obtaining submodule is configured to obtain a relative position relationship between the second positioning marker and the first marker in the registration assembly. The third spatial coordinate submodule is configured to obtain a fifth spatial coordinate of the second positioning marker in a real three-dimensional space. The fourth spatial coordinate obtaining submodule is configured to determine a first spatial coordinate of the first marker in the real three-dimensional space in the registration assembly based on the fifth spatial coordinate and the relative position relationship.

16. The apparatus of claim 15, wherein, The third spatial coordinate obtaining submodule is specifically configured to: obtain a second target image containing the second positioning marker, which is collected by a second image collection device; identify a feature point coordinate of the second positioning marker in the second target image; and obtain second pose information of the second positioning marker relative to the second image collection device according to the identified feature point coordinate. The fifth spatial coordinate of the second positioning marker is determined based on the second pose information.

17. An electronic device, comprising: The apparatus includes a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory are in communication with each other through the communication bus. The memory is configured to store a computer program. The processor is configured to execute the program stored on the memory to implement the method steps of any one of claims 1-8.

18. A computer-readable storage medium, characterized in that, The computer program is stored in the computer readable storage medium, and the computer program is executed by the processor to implement the method steps of any one of claims 1-8.

Images