Sorting method and device for calibration characteristics, and calibration method and device

Abstract

The invention discloses a calibration feature sorting method and device, a calibration method and device of an image shooting device, computer equipment, a computer readable storage medium and a computer program product. The calibration feature sorting method comprises the following steps that a calibration piece image is obtained, the calibration piece image is configured to be an image obtained by shooting a calibration piece provided with a plurality of markers, and the calibration piece image comprises a plurality of calibration features corresponding to the markers; identifying a plurality of calibration features in the calibration part image to determine a feature outer contour, wherein the feature outer contour is configured as an outer contour corresponding to the plurality of calibration features; determining image position information corresponding to each marker based on a conversion relation between the feature outer contour and a reference contour so as to sort the plurality of calibration features; wherein the reference contour is configured to correspond to outer contours of a plurality of markers.

Description

Technical Field

[0001] The present application relates to the field of computer vision technology, and in particular to a method and device for sorting calibration features, a method and device for calibrating an image capture device, a computer device, a computer-readable storage medium, and a computer program product. Background Art

[0002] Computer vision technology uses image capture devices (such as visual cameras, depth cameras, and X-ray machines) and computer equipment to identify, track, and measure targets. This technology has a wide range of applications in various fields, such as medical imaging, autonomous driving, industrial automation, security monitoring, and facial recognition.

[0003] Before using an image capture device, it is usually necessary to calibrate it. The main purpose of calibrating an image capture device is to obtain parameters such as the device's intrinsic and extrinsic parameters and lens distortion. This allows computers to improve the accuracy of image processing when using these calibrated parameters to process images captured by the image capture device.

[0004] Typically, calibration is performed using calibration points on a calibration plate. However, the positional relationship between the calibration features in the image and the calibration points on the plate cannot be matched one-to-one, resulting in calibration failure. Therefore, how to match the positional relationship between the calibration features in the image and the calibration points on the plate is a technical problem that needs to be solved by those skilled in the art. Summary of the Invention

[0005] In view of the shortcomings of the related technologies mentioned above, the purpose of the present application is to provide a method and device for sorting calibration features, a method and device for calibrating an image capture device, a computer device, a computer-readable storage medium, and a computer program product, which are used to solve the technical problem of difficulty in matching the positional relationship between each calibration feature in the image acquired by the image capture device and each calibration point on the calibration plate.

[0006] To achieve the above-mentioned purpose and other related purposes, the first aspect of the present application provides a method for sorting calibration features, comprising the following steps: acquiring a calibration part image, wherein the calibration part image is configured as an image obtained by photographing a calibration part equipped with multiple markers, and the calibration part image includes multiple calibration features corresponding to the multiple markers; identifying the multiple calibration features in the calibration part image to determine a feature outer contour, wherein the feature outer contour is configured as an outer contour corresponding to multiple calibration features; determining the image position information corresponding to each marker based on a conversion relationship between the feature outer contour and a reference contour, so as to sort the multiple calibration features; wherein the reference contour is configured as an outer contour corresponding to multiple markers.

[0007] The second aspect of the present application provides a calibration method for an image capture device, comprising the following steps: sorting a plurality of calibration features on an image of a calibration part, wherein the sequence of the sorted plurality of calibration features is consistent with the sequence of a plurality of markers configured on the calibration part; wherein the sorting method can be configured as the calibration feature sorting method as described in any embodiment disclosed in the first aspect of the present application; and performing internal or external parameter calibration on the image capture device based on the plurality of calibration features and a preset calibration algorithm.

[0008] According to a third aspect of the present application, there is provided a calibration feature sorting device, comprising: an image acquisition module for acquiring a calibration part image, wherein the calibration part image is configured as an image obtained by photographing a calibration part equipped with multiple markers, and the calibration part image includes multiple calibration features corresponding to the multiple markers; a feature outer contour determination module for identifying the multiple calibration features based on the calibration part image to determine the feature outer contour, wherein the feature outer contour is configured as an outer contour corresponding to the multiple calibration features; and a sorting module for determining the image position information corresponding to each marker based on a conversion relationship between the feature outer contour and a reference contour, so as to sort the multiple calibration features based on the image position data corresponding to each marker; wherein the reference contour is configured as an outer contour corresponding to the multiple markers.

[0009] The fourth aspect of the present application provides a calibration device for an image capture device, comprising: a sorting module configured as a sorting device as described in any embodiment disclosed in the third aspect of the present application; a calibration module, performing internal or external parameter calibration on the image capture device based on multiple calibration features involved in the sorting module and a preset calibration algorithm.

[0010] The fifth aspect of the present application provides a computer device, comprising: a storage device for storing at least one program; a processing device, connected to the storage device, for calling the at least one program from the storage device and executing it to implement the sorting method as described in any embodiment disclosed in the first aspect of the present application, or the calibration method as described in any embodiment disclosed in the second aspect of the present application.

[0011] The sixth aspect of the present application provides a computer-readable storage medium storing at least one program, which, when called and executed by a computer processor, implements the sorting method as described in any embodiment disclosed in the first aspect of the present application, or the calibration method as described in any embodiment disclosed in the second aspect of the present application.

[0012] The seventh aspect of the present application provides a computer program product, which, when run on a computer, enables the computer to execute the sorting method described in any embodiment disclosed in the first aspect of the present application, or the calibration method described in any embodiment disclosed in the second aspect of the present application.

[0013] In summary, the calibration feature sorting method and device, image capture device calibration method and device, computer equipment, computer-readable storage medium, and computer program product provided in the present application determine the image position information corresponding to each marker by establishing a conversion relationship between the reference contour of each marker on the calibration part and the characteristic outer contour of each calibration feature on the calibration part image, thereby establishing a one-to-one positional relationship between the marker and the calibration feature, and then sorting the calibration features on the calibration part image to ensure the feasibility of calibrating the image capture device. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] The specific features of the present application are set forth in the appended claims. The features and advantages of the present invention can be better understood by referring to the exemplary embodiments described in detail below and the accompanying drawings. A brief description of the drawings is as follows:

[0015] Figure 1 and Figure 2 Shown are schematic diagrams of the arrangement of multiple markers on the calibration piece in different embodiments of the present application.

[0016] Figure 3 Shown is a flow chart of a method for sorting calibration features in one embodiment of the present application.

[0017] Figure 4 Shown is a schematic diagram of a calibration part image in one embodiment of the present application.

[0018] Figure 5 Shown is a schematic diagram of a process for determining a feature outer contour in one embodiment of the present application.

[0019] Figure 6 Shown is a schematic diagram of a process for sorting multiple calibration features based on image position information in one embodiment of the present application.

[0020] Figure 7 FIG. 1 is a flow chart of a calibration method for an image capture device according to an embodiment of the present application.

[0021] Figure 8 Shown is a module block diagram of a sorting device for calibrating features in one embodiment of the present application.

[0022] Figure 9 Shown is a module block diagram of a calibration device for an image capture device in one embodiment of the present application.

[0023] Figure 10 Shown is a structural diagram of a computer device in one embodiment of the present application. DETAILED DESCRIPTION

[0024] The following describes the implementation of the present application through specific embodiments. People familiar with this technology can easily understand other advantages and effects of the present application from the contents disclosed in this specification.

[0025] The present application is further described in detail below in conjunction with the accompanying drawings and specific embodiments. The technical solutions in the embodiments of the present application are clearly and completely described. Obviously, the described embodiments are only embodiments of a part of the present application, rather than all embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application. "One embodiment", "embodiment" or similar wordings mentioned throughout this specification mean that the specific features, structures or characteristics described together with the embodiment are included in at least one embodiment of the present application. Therefore, throughout this specification, the appearance of the phrases "in one embodiment", "in an embodiment" and similar wordings may (but not necessarily) relate to the same embodiment.

[0026] As described in the background art, it is difficult to match the positional relationship between each calibration feature in the image and each calibration point on the calibration plate during calibration, which will result in the inability to perform calibration. In view of this, some embodiments provided in this application disclose a calibration feature sorting method and device, an image capture device calibration method and device, a computer device, a computer-readable storage medium, and a computer program product. By establishing a conversion relationship between the reference contour of each marker on the calibration part and the characteristic outer contour of each calibration feature on the calibration part image, the image position information corresponding to each marker is determined, thereby establishing a one-to-one positional relationship between the marker and the calibration feature, and then sorting the calibration features on the calibration part image, thereby ensuring the feasibility of calibrating the image capture device.

[0027] In an embodiment of the present application, a calibration component is configured to calibrate the internal and / or external parameters of an image capture device used to capture the calibration component. This means that the calibration component is used to calibrate the image capture device, and can be specifically used to calibrate the internal or external parameters of the image capture device, or can be used to calibrate both the internal and external parameters of the image capture device. In examples where the calibration component is configured solely to calibrate internal or external parameters, it can be referred to as an external parameter calibration component or an internal parameter calibration component, respectively.

[0028] The calibration piece is provided with a plurality of markers, and the plurality of markers are arranged so as not to be distributed on the same straight line. In order to ensure that the plurality of markers are not distributed on the same straight line, the number of markers is at least three. Figure 1 and Figure 2 , respectively showing the arrangement diagrams of multiple markers on the calibration piece in different embodiments of the present application. Figure 1 In the example shown, there are four markers on the calibration piece. Figure 2 In the example shown, nine markers are arranged on the calibration piece. The calibration points are arranged, for example, according to a preset rule, which represents the arrangement of multiple calibration points on the calibration piece. For example, the preset rule may be that the calibration points are arranged in rows and columns, and the spacing between adjacent calibration points in the same direction (for example, the same row or the same column) may be the same or different. The preset rule may also be that the outer contour of each calibration point is a specified shape, for example, Figure 1 The outer contour shown in is a triangle. Figure 2 The outer contour shown in is a quadrilateral.

[0029] It should be noted that in the subsequent embodiments, Figure 2 The calibration parts shown in the figure are used as an example to illustrate the sorting method described in this application. For the convenience of description and understanding, Figure 2 Each black dot in the figure represents a marker, where a1 represents the first marker on the calibration piece, a2 represents the second marker on the calibration piece, a3 represents the third marker on the calibration piece, and so on, a9 represents the ninth marker on the calibration piece. Figure 2 The markers in are respectively called marker a1, marker a2, marker a3, ..., marker a9.

[0030] Furthermore, the marker is configured as a calibration point formed on the calibration piece. In one example, the calibration point is configured as a physical object, which may be, for example, a spherical steel ball. An adaptation space for embedding the spherical steel ball may be provided on the calibration piece, and the spherical steel ball is configured in the space. In another example, the calibration point is configured as a pattern mark, which may be, for example, a printed picture pasted on the calibration piece. The calibration piece may be configured as a calibration plate. The calibration point may have a specific geometric shape, for example, the calibration point may be configured as a regular shape such as a sphere, a cube, a circle, or a square.

[0031] In one embodiment, the image capture device is configured to convert light energy reflected or transmitted by each measurement point of an object captured within its viewing angle in physical space into an image having a corresponding pixel resolution. The measurement points are regions on the physical object corresponding to pixel positions in the image based on light reflection or light transmission principles.

[0032] In one embodiment, the image capturing device acquires an image based on light energy reflected by a physical object, and the image capturing device includes but is not limited to at least one of the following: a monocular camera device, a binocular camera device, a depth camera device, etc.

[0033] In one embodiment, the image capturing device acquires an image based on light energy transmitted by a physical object, and the image capturing device comprises medical imaging equipment. Such medical imaging equipment includes, but is not limited to, CT scanners and X-ray machines. In the subsequent embodiments, the image capturing device is configured as an X-ray machine, which should not be construed by those skilled in the art as limiting the present application.

[0034] The X-ray machine can be used in a surgical navigation system. The surgical navigation system is used to provide lesion location information and / or surgical path navigation information to the surgeon or robotic arm performing the surgery during surgery. In one embodiment, the surgical navigation system, in addition to including the X-ray machine, may also include an optical positioning tracking device (e.g., an NDI camera), a computer device, and a robot. The computer device is communicatively connected to the optical positioning tracking device, the X-ray machine, and the robot. For example, the computer device can match X-ray images captured by the X-ray machine during surgery with pre-operative image data (e.g., CT data). The optical positioning tracking device can use tracking devices respectively installed on the X-ray machine, the robot, and the target object to track the positions of the three in real time. In other words, the optical positioning tracking device can determine the coordinate positions of the X-ray machine, the robot, and the target object in their respective coordinate systems in real time. The robot can, for example, be a robotic arm that grips surgical instruments, the X-ray machine can be a C-arm X-ray machine, and the target object can be, for example, a human body, an animal body, or a model.

[0035] In some embodiments, the present application provides a method for sorting calibration features, which can be executed by a computer device equipped with a calibration feature sorting device. The sorting device can also be configured as a module in a calibration device, which can be configured to execute in the computer device. Of course, the calibration device can also be further configured as a module in a surgical path navigation device, which can be configured to execute in the computer device. The sorting device or the calibration device is a software tool or software module that can process data, and it processes data using the hardware devices in the computer device and the operating environment provided by the operating system.

[0036] In one embodiment, the computer device may be the computer device configured in the surgical navigation system described in the aforementioned embodiment. The computer device may be configured as an electronic device, that is, the electronic device performs the calibration feature sorting method. For example, the electronic device includes a desktop computer, a laptop computer, a tablet computer, a smart TV, a smartphone, and a tablet computer. The electronic device may also be an electronic device composed of a host computer with multiple virtual machines and a human-computer interaction device corresponding to each virtual machine (such as a touch screen, a keyboard, and a mouse).

[0037] In one embodiment, the computer device can be configured as a server, that is, the server executes the method for sorting the calibration features. The server can be deployed on one or more physical servers based on various factors such as function and load. In some examples, the server can be a cloud-based server, which refers to a cloud computing platform provided by a cloud computing provider. The cloud computing platform can provide services such as IaaS (Infrastructure as a Service), PaaS (Platform as a Service), and SaaS (Software as a Service). The cloud computing platform includes public cloud, private cloud, and hybrid cloud. In some examples, the server can be composed of a distributed or centralized server cluster. For example, the server cluster is composed of at least one physical server. Each physical server is configured with multiple virtual servers, each virtual server runs at least one functional module in the system, and the virtual servers communicate with each other through a network.

[0038] See also Figure 3 , which is a flow chart of a method for sorting calibration features in one embodiment of the present application. As shown in the figure, the method includes steps S110, S120, and S130. The following describes various embodiments using the method for sorting calibration features executed by a computer device as an example.

[0039] In step S110 , the computer device acquires an image of a calibration part.

[0040] In one embodiment, the calibration part image is configured as an image obtained by photographing a calibration part configured with multiple markers. Figure 4 Combined with Figure 2 , Figure 4 The diagram shows a schematic diagram of a calibration part image in one embodiment of the present application, wherein the image capturing device is Figure 2 The calibration piece shown is photographed and obtained Figure 4 It should be understood that since the image capture device is an imaging device and is also affected by its own parameters or installation position, the calibration part image is not equivalent to the real calibration part, for example Figure 4 In the figure, the calibration part image is obtained by shooting the calibration part at a certain viewing angle, and is rotated relative to the calibration part.

[0041] Accordingly, the calibration image includes multiple calibration features corresponding to the multiple markers. The calibration feature corresponding to a certain marker can be an image feature composed of all pixel points representing the marker in the calibration image, or it can be an image feature composed of edge pixel points representing the marker in the calibration image. Figure 4 In the example shown, each black dot represents a calibration feature. Figure 4 The calibration features in are marked as calibration feature b1, calibration feature b2, calibration feature b3, ..., calibration feature b9.

[0042] Please continue reading Figure 3 In step S120, the computer device identifies multiple calibration features in the calibration part image to determine the feature outer contour.

[0043] The feature outer contour is configured to correspond to the outer contour of multiple calibration features, which refers to the geometric shape formed by connecting the outermost calibration features and enclosing all the calibration features. The calibration features used for connecting the lines constitute the corners of the geometric shape, and are therefore also called corner features. Figure 4 As shown in the example, the line n connecting the calibration features b1, b3, b4, and b9 can enclose all the calibration features b1 to b9. The calibration features b1, b3, b4, and b9 are corner features, and the line n is the feature outer contour of the calibration features b1 to b9, and the geometric shape formed by them is a quadrilateral. Of course, in other examples, the feature outer contour can be any shape, depending on the position of each calibration feature. In the embodiment described in the present application, the feature outer contour passes through the center of each corner feature. In other embodiments, the feature outer contour can also be tangent to each corner feature.

[0044] See also Figure 5 , which is a schematic diagram of a process for determining a feature outer contour in one embodiment of the present application. As shown in the figure, step S120 includes step S121 and step S122.

[0045] In step S121 , the computer device performs feature recognition on the calibration part image to determine image position information of a plurality of calibration features.

[0046] In one embodiment, the computer device performs feature recognition on the calibration part image using a traditional feature extraction method based on certain inherent features of the calibration part image to determine the image position information of multiple calibration features. In another embodiment, the computer device performs feature recognition on the calibration part image using a deep learning method based on a deep learning model trained on a large number of images to determine the image position information of multiple calibration features.

[0047] It should be noted that the image capture device is Figure 2 The calibration piece shown is photographed and obtained Figure 4 In the calibration part image shown, although the computer device can identify the calibration features b1 to b9 located on the calibration part image, it cannot form a one-to-one correspondence with the markers a1 to a9 located on the calibration part. For example, the first calibration feature b1 in the calibration part image does not correspond to the first marker a1 on the calibration part.

[0048] Please continue reading Figure 5 In step S122, the computer device selects corner features based on the position information of multiple calibration features to form the feature outer contour. Figure 4 In the example shown, if the computer device wants to obtain the feature outer contour n, it must be able to select the corner point features that form the feature outer contour from the nine calibration features, namely, calibration feature b1, calibration feature b2, calibration feature b3, ..., calibration feature b9, that is, calibration feature b1, calibration feature b3, calibration feature b4 and calibration feature b9.

[0049] In one embodiment, the computer device selects corner features from among the multiple calibration features based on their positional information to form the feature outer contour, including determining the feature outer contour using a convex hull algorithm. Specifically, an image coordinate system is first established in the calibration part image, and the calibration feature with the smallest ordinate is selected from all calibration features. If multiple calibration features have minimum ordinates, the calibration feature with the smallest abscissa is selected as a corner feature on the convex hull. Next, the remaining calibration features are sorted by polar angle based on the first selected corner feature, with the smaller polar angle being ranked first. For identical polar angles, the distances to the corner features are compared, with the closer distance being prioritized. Next, the computer device selects points from each sorted calibration feature and checks whether they are corner features and whether the line connecting all corner features forms a convex polygon. If the addition of a calibration feature would destroy the convexity of the convex polygon, the calibration feature cannot be used as a corner feature. Finally, the computer device constructs a convex hull based on the corner features, which constitutes the feature outer contour.

[0050] Please continue reading Figure 3 In step S130, the computer device determines the image position information corresponding to each marker based on the conversion relationship between the feature outer contour and a reference contour to sort the multiple calibration features.

[0051] The reference outline is configured as the outer outline of the corresponding multiple markers, which refers to the geometric shape formed by connecting the outermost markers to enclose all the markers. Figure 1On the calibration piece shown, the line k between the three outer markers can surround all four markers, so the line k is the reference contour. In this example, the reference contour k is a triangle. Figure 2 On the calibration component shown, the line m connecting markers a1, a2, a6, and a9 encloses all markers a1 through a9, thus forming the reference contour. In this example, the reference contour m is a quadrilateral. Of course, in other examples, the reference contour can be any shape, depending on the position of the markers. In this example, the reference contour passes through the center of the outermost markers. In other examples, the reference contour can also be circumscribed to the outermost markers.

[0052] It should be noted that although the calibration image is not equivalent to the real calibration object, there is generally an affine transformation relationship between the two. Transformations such as translation, rotation, and scaling do not produce changes in geometric properties. That is, the geometric shape of the feature outer contour corresponds to that of the reference contour. For example, Figure 2 and Figure 4 As shown, when the reference contour m is a quadrilateral, the characteristic outer contour n is also a quadrilateral. In other examples, when the reference contour is a pentagon, the characteristic outer contour is also a pentagon.

[0053] In order to ensure that the characteristic outer contour and the reference contour are polygonal, as mentioned above, the multiple markers are configured to be distributed on different lines. This means that the multiple markers are configured to be distributed on different lines in the calibration part, so as to ensure that the shape of the reference contour on the calibration part can form a closed polygon instead of a straight line. Correspondingly, the characteristic outer contour of the calibration feature on the calibration part image captured by the image capture device is also a polygon. For example, Figure 1 In the example shown, when the number of the markers is four, at most two markers can be collinear, that is, the number of sides of the closed polygon formed by the reference contour and the feature outer contour is at least three, which can be a triangle or a quadrilateral.

[0054] In one embodiment, the reference contour may be pre-stored in the computer device. When determining the conversion relationship between the feature outer contour and the reference contour, the computer device can directly call the reference contour. In another embodiment, the reference contour may be determined by the computer device based on the physical position information of the multiple markers. In other words, step S130 includes the step of determining the reference contour based on the physical position information of the multiple markers.

[0055] The physical location information refers to the real location information of each marker on the calibration object in the real world coordinate system. In one embodiment, the physical location information includes the sequence and physical coordinates of the marker. For example, the physical location information of a marker can be described as a i (x i ,y i ), where i represents the sequence of the marker, (x, y) represents the physical coordinates of the marker, and for ease of representation, it is briefly described as a i ,by Figure 2 Taking the example shown, the physical location information of marker a1 is a1(x1, y1), the physical location information of marker a2 is a2(x2, y2), the physical location information of marker a3 is a3(x3, y3), and so on, the physical location information of marker a9 is a9(x9, y9).

[0056] In one embodiment, the physical position information of the plurality of markers is determined based on the size information of the calibration plate and the size information between the markers. Figure 2 Taking the calibration plate shown as an example, if the spacing between any two adjacent calibration points in the row direction is equal to the spacing between any two adjacent calibration points in the column direction and is 1, and if the actual world coordinate system is established with marker a6 as the origin, then the physical position information of markers a1 to a9 is a1(2,2), a2(3,2), a3(1,1), a4(2,1), a5(3,1), a6(0,0), a7(1,0), a8(2,0), a9(3,0). For example, a computer device can use a convex hull algorithm to determine the reference contour m based on the physical position information of markers a1 to a9. The convex hull algorithm is similar to the method for determining the feature outer contour described above and will not be repeated here.

[0057] Furthermore, in step S130 , the computer device further executes a step of performing contour matching between the feature outer contour and the reference contour to determine a conversion relationship between the two.

[0058] In one embodiment, a matching algorithm can be used to determine the conversion relationship between the two. For example, the findHomography method, similarity measurement, singular value decomposition, least squares method, etc. can be used to determine the conversion relationship between the two. It should be noted that before determining the conversion relationship between the two, the feature outer contour and the reference contour can also be normalized to eliminate the influence of factors such as scale and rotation to ensure that the shape information of the feature outer contour and the reference contour can match.

[0059] The transformation relationship between the characteristic outer contour and the reference contour described in step S130 can be used to determine the image position information corresponding to each marker. The transformation relationship can be, for example, a geometric transformation parameter or a transformation matrix. For ease of explanation, the transformation relationship between the reference contour and the characteristic outer contour will be represented by H.

[0060] See also Figure 6 , which is a flow chart showing the process of sorting multiple calibration features based on image position information in one embodiment of the present application. Figure 6 As shown, step S130 includes step S131 and step S132.

[0061] In step S131 , the computer device obtains the image position information corresponding to each marker based on the physical position information of the multiple markers and the conversion relationship.

[0062] In one embodiment, the physical position information of each marker is multiplied by the conversion relationship to obtain the image position information corresponding to each marker. For example, the physical position information of marker a1 is multiplied by the conversion relationship H to obtain the image position information corresponding to marker a1, the physical position information of marker a2 is multiplied by the conversion relationship H to obtain the image position information corresponding to marker a2, the physical position information of marker a3 is multiplied by the conversion relationship H to obtain the image position information corresponding to marker a3, and so on. The physical position information of marker a9 is multiplied by the conversion relationship H to obtain the image position information corresponding to marker a9.

[0063] In step S132 , the image position information corresponding to each marker is matched with the image position information of each calibration feature to determine the calibration features corresponding to each marker, so as to sort the calibration features.

[0064] The image position information of the calibration feature refers to pixel position information. In one embodiment, the pixel position information includes a pixel sequence and pixel coordinates of the calibration feature. The pixel sequence and pixel coordinates can be found in the description of the physical position information in the above embodiment and are not repeated here.

[0065] In one embodiment, the matching refers to a computer device calculating a degree of match between the image position information corresponding to each marker and the image position information of each calibration feature, and determining the calibration feature corresponding to each marker based on the result of the matching degree calculation. In one example, the matching degree calculation may be calculating the positional difference between the image position information of a first marker and the image position information of each calibration feature, selecting the calibration feature whose positional difference is less than a preset distance as the calibration feature matching the first marker, then continuing to calculate the positional difference between the image position information of a second marker and the image position information of each calibration feature, selecting the calibration feature whose positional difference is less than the preset distance as the calibration feature matching the second marker, and so on, until matching calibration features are found for all markers.

[0066] by Figure 4 and Figure 2As shown in the figure as an example, according to the method for determining the image position information of the markers in the aforementioned embodiment, the image position information corresponding to marker a1 is represented as a1×H. The computer device can match the image position information corresponding to marker a1 with the image position information of calibration features b1 to b9 respectively, and can determine that the position difference between marker a1 and calibration feature b4 is less than the preset distance, that is, it can be determined that calibration feature b4 corresponds to marker a1. The image position information corresponding to marker a2 is represented as a2×H. The computer device can match the image position information corresponding to marker a2 with the image position information of calibration features b1 to b9 respectively, and can determine that the position difference between marker a2 and calibration feature b1 is less than the preset distance, that is, it can be determined that calibration feature b1 corresponds to marker a2. The image position information corresponding to marker a3 is represented as a3×H. Based on this image position information, the computer device can match it with the image position information of calibration features b1 to b9. This can determine that the positional difference between marker a3 and calibration feature b7 is less than a preset distance, indicating that calibration feature b7 corresponds to marker a3. The image position information corresponding to marker a4 is represented as a4×H. Based on this image position information, the computer device can match it with the image position information of calibration features b1 to b9. This can determine that the positional difference between marker a4 and calibration feature b5 is less than a preset distance, indicating that calibration feature b5 corresponds to marker a4. The image position information corresponding to marker a5 is represented as a5×H. Based on this image position information, the computer device can match it with the image position information of calibration features b1 to b9. This can determine that the positional difference between marker a5 and calibration feature b2 is less than a preset distance, meaning that calibration feature b2 corresponds to marker a5. The image position information corresponding to marker a6 is represented as a6×H. Based on this image position information, the computer device can match it with the image position information of calibration features b1 to b9. This can determine that the positional difference between marker a6 and calibration feature b9 is less than a preset distance, meaning that calibration feature b9 corresponds to marker a6. The image position information corresponding to marker a7 is expressed as a7×H. The computer device can match the image position information corresponding to marker a7 with the image position information of calibration features b1 to b9 respectively. It can be determined that the position difference between marker a7 and calibration feature b8 is less than the preset distance, that is, it can be determined that calibration feature b8 corresponds to marker a7.The image position information corresponding to marker a8 is represented as a8×H. Based on this image position information, the computer device can match the image position information of calibration features b1 through b9, respectively. This can determine that the positional difference between marker a8 and calibration feature b6 is less than a preset distance, meaning that calibration feature b6 corresponds to marker a8. The image position information corresponding to marker a9 is represented as a9×H. Based on this image position information, the computer device can match the image position information of marker a9 with the image position information of calibration features b1 through b9, respectively. This can determine that the positional difference between marker a9 and calibration feature b3 is less than a preset distance, meaning that calibration feature b3 corresponds to marker a9.

[0067] Furthermore, the computer device may sort the calibration features according to the sequence of each marker, that is, the computer device will re-sort the calibration features according to the sequence of the marker. For ease of explanation, the re-sorted calibration features are represented by c i Indicates that Figure 2 and Figure 4 As shown in the figure, in order to correspond to each marker, the calibration feature b4 is represented by c1, the calibration feature b1 is represented by c2, the calibration feature b7 is represented by c3, the calibration feature b5 is represented by c4, the calibration feature b2 is represented by c5, the calibration feature b9 is represented by c6, the calibration feature b8 is represented by c7, the calibration feature b6 is represented by c8, and the calibration feature b3 is represented by c9. Figure 4 After sorting the calibration features shown, the calibration features corresponding to marker a1, marker a2, marker a3, marker a4, marker a5, marker a6, marker a7, marker a8, and marker a9 are c1, c2, c3, c4, c5, c6, c7, c8, and c9 respectively.

[0068] To sum up, the calibration feature sorting method disclosed in the present application determines the image position information corresponding to each marker by establishing a conversion relationship between the reference contour of each marker on the calibration part and the characteristic outer contour of each calibration feature on the calibration part image, and establishes a one-to-one positional relationship between the marker and the calibration feature by matching the image position information corresponding to each marker with the image position information of each calibration feature, and then sorts the calibration features on the calibration part image to ensure the feasibility of calibration of the image capture device.

[0069] The present application also provides a calibration method for an image capture device, which can be executed by a computer device equipped with a surgical navigation system. The specific structure and function of the computer device and the surgical navigation system can be found in the description of the calibration feature sorting method above and will not be repeated here.

[0070] See also Figure 7 , which is a flow chart of a calibration method for an image pickup device in one embodiment of the present application. As shown in the figure, the calibration method for an image pickup device includes step S210 and step S220.

[0071] In step S210 , the computer device sorts a plurality of calibration features on a calibration part image, and the sequence of the sorted plurality of calibration features is consistent with the sequence of the plurality of markers configured on the calibration part.

[0072] The method for sorting multiple calibration features on a calibration part image can refer to the aforementioned method for sorting multiple calibration features on a calibration part image. Figures 1 to 6 The description of any related embodiments will not be repeated here.

[0073] In step S220, the computer device performs internal or external parameter calibration on the image capture device based on the multiple calibration features and a preset calibration algorithm.

[0074] In one embodiment, a computer device performs internal parameter calibration on the image capture device based on the multiple calibration features and a preset calibration algorithm. The preset calibration algorithm may be, for example, the Zhang Zhengyou calibration method. Specifically, based on the acquired image position information of the multiple calibration features and the physical position information of each marker on the calibration object, the Zhang Zhengyou calibration method is used to determine the internal parameter matrix of the image capture device, thereby determining the focal length (f x ,f y ) and principal point coordinates (c x ,c y ).

[0075] In another embodiment, a computer device performs extrinsic calibration on the image capture device based on the multiple calibration features and a preset calibration algorithm. Specifically, based on an intrinsic parameter matrix obtained through the intrinsic parameter calibration, as well as the acquired image position information of the multiple calibration features, the physical position information of each marker on the calibration object, and the size information of the calibration object, an extrinsic parameter matrix of the image capture device can be determined, thereby performing extrinsic calibration on the image capture device.

[0076] This application also provides a device for sorting calibration features. In an embodiment, the device is deployed, for example, in a surgical navigation system. The device can be a software tool or software module capable of processing data, utilizing hardware devices in a computer device and an operating environment provided by an operating system to process the data. The computer device is configured in the surgical navigation system. The specific structure and function of the computer device and the surgical navigation system can be found in the previous description and will not be further elaborated here.

[0077] See also Figure 8, which is a module block diagram of a calibration feature sorting device in one embodiment of the present application. As shown in the figure, the calibration feature sorting device 1 includes an image acquisition module 10, a feature outer contour determination module 11, and a sorting module 12. The image acquisition module 10 is used to acquire a calibration part image, wherein the calibration part image is configured as an image obtained by photographing a calibration part equipped with multiple markers, and the calibration part image includes multiple calibration features corresponding to the multiple markers. The feature outer contour determination module 11 is used to identify the multiple calibration features based on the calibration part image to determine the feature outer contour, wherein the feature outer contour is configured as the outer contour corresponding to the multiple calibration features. The sorting module 12 is used to determine the image position information corresponding to each marker based on the conversion relationship between the feature outer contour and a reference contour, so as to sort the multiple calibration features based on the image position data corresponding to each marker, wherein the reference contour is configured as the outer contour corresponding to the multiple markers.

[0078] The image acquisition module 10, the feature outer contour determination module 11, and the sorting module 12 in the calibration feature sorting device 1 cooperate to execute the calibration feature sorting method disclosed in any embodiment of the present application according to the modules described above. Figures 1 to 6 Any embodiments described in related manner will not be described in detail here.

[0079] This application also provides a calibration device for an image capture device. In an embodiment, the calibration device for an image capture device is deployed, for example, within a surgical navigation system and executed by a computer device configured within the surgical navigation system. The specific structure and function of the computer device and the surgical navigation system can be found in the description of the calibration feature sorting method above and will not be further elaborated here.

[0080] See also Figure 9 , which is a block diagram of a calibration device for an image pickup device in one embodiment of the present application. As shown in the figure, the calibration device 2 for the image pickup device includes a sorting module 20 and a calibration module 21. The sorting module 20 is configured as a sorting device for the calibration features disclosed in any embodiment of the present application, see Figure 8 The calibration module 21 performs internal or external calibration on the image capture device based on the multiple calibration features involved in the sorting module and a preset calibration algorithm.

[0081] This application also provides a computer device. In an embodiment, the computer device can be deployed in a surgical navigation system to implement the method for sorting calibration features or the method for calibrating an image capture device described in any of the above embodiments. In one embodiment, the computer device is a device capable of performing digital calculations, logical processing, and information processing on data, including but not limited to: a personal computer, a server, a server cluster, an intelligent terminal, a cloud-based server system, etc.

[0082] See also Figure 10 , which is a schematic diagram of the structure of a computer device in one embodiment of the present application, wherein the computer device 3 includes a storage device 30 and a processing device 31 connected to the storage device 30. Furthermore, the computer device also includes an interface device 32.

[0083] In some embodiments, the storage device 30 is configured to store at least one program, which is executable by the processing device 31 to coordinate the storage device 30 with implementing the calibration feature sorting method or image capture device calibration method described in any of the above embodiments. Here, the storage device 30 includes, but is not limited to, read-only memory (ROM), random access memory (RAM), and non-volatile RAM (NVRAM). For example, the storage device 30 includes a flash memory device or other non-volatile solid-state storage device. In certain embodiments, the storage device 30 may also include memory remote from one or more processing devices 31, such as a network attached storage (NAS) accessed via RF circuitry or an external port and a communication network, wherein the communication network may be the Internet, one or more intranets, local area networks, wide area networks, storage area networks, or a suitable combination thereof. A memory controller may control access to the memory by other components of the device, such as the CPU and peripheral interfaces.

[0084] In some embodiments, the processing device 31 includes one or more processors. The processing device 31 is operable to perform data read and write operations with the storage device 30. The processing device 31 includes one or more general-purpose microprocessors, one or more application-specific processors (ASICs), one or more digital signal processors (DSPs), one or more field programmable gate arrays (FPGAs), or any combination thereof.

[0085] In some embodiments, the interface device 32 includes at least one interface unit, each of which is used to output a visual interface, receive human-computer interaction events generated by technicians' operations, and so on. For example, the interface device 32 includes, but is not limited to, a serial interface such as an HDMI interface or a USB interface, or a parallel interface. In one embodiment, the interface device 32 also includes a network communication unit, which is a device that transmits data via a wired or wireless network. Examples include, but are not limited to, an integrated circuit containing a network card, a local area network module such as a WiFi module or a Bluetooth module, and a wide area network module such as a mobile network.

[0086] The present application also provides a computer-readable storage medium storing at least one program, which, when called and executed by a computer processor, implements the calibration feature sorting method or image capture device calibration method as described in any of the above embodiments.

[0087] The present application also provides a computer program product. When the computer program product is run on a computer, the computer is caused to execute the above-mentioned related steps to implement the calibration feature sorting method or image capture device calibration method in any of the above-mentioned embodiments.

[0088] If the method is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes a number of instructions for enabling a robot equipped with the storage medium to execute all or part of the steps of the method described in each embodiment of the present application.

[0089] In the embodiments provided herein, the computer storage medium may include a read-only memory, a random access memory, an EEPROM, a CD-ROM or other optical disk storage device, a magnetic disk storage device or other magnetic storage device, a flash memory, a USB flash drive, a mobile hard disk, or any other medium that can be used to store desired program code in the form of instructions or data structures and can be accessed by a computer. In addition, any connection can be appropriately referred to as a computer-readable medium. For example, if the instruction is sent from a website, server or other remote source using a coaxial cable, a fiber optic cable, a twisted pair, a digital subscriber line (DSL) or wireless technologies such as infrared, radio and microwaves, the coaxial cable, fiber optic cable, twisted pair, DSL or wireless technologies such as infrared, radio and microwaves are included in the definition of the medium. However, it should be understood that computer storage media and data storage media do not include connections, carriers, signals or other temporary media, but are intended to be non-temporary, tangible storage media. Disk and disc, as used in this application, includes compact disc (CD), laser disc, optical disc, digital versatile disc (DVD), floppy disk and Blu-ray disc where disks usually reproduce data magnetically, while discs reproduce data optically with lasers.

[0090] In one or more exemplary aspects, the functions described in the method for sorting calibration features or the method for calibrating an image capture device described herein may be implemented in hardware, software, firmware, or any combination thereof. When implemented in software, these functions may be stored or transmitted as one or more instructions or codes on a computer-readable medium. The steps of the method or algorithm disclosed herein may be embodied in a processor-executable software module, wherein the processor-executable software module may be located on a tangible, non-transitory computer storage medium. A tangible, non-transitory computer storage medium may be any available medium that can be accessed by a computer.

[0091] The flowcharts and block diagrams in the accompanying drawings described in this application illustrate the possible implementation architecture, functions and operations of the system, method and computer program product according to various embodiments of the present application. Based on this, each box in the flowchart or block diagram can represent a module, program segment, or a part of code, and the module, program segment, or a part of code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order than that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart, and the combination of the boxes in the block diagram and / or flowchart can be implemented by a dedicated hardware-based system that performs the specified function or operation, or can be implemented by a combination of dedicated hardware and computer instructions.

[0092] In summary, the calibration feature sorting method and device, image capture device calibration method and device, computer equipment, computer-readable storage medium, and computer program product disclosed in the present application determine the image position information corresponding to each marker by establishing a conversion relationship between the reference contour of each marker on the calibration part and the characteristic outer contour of each calibration feature on the calibration part image, and establish a one-to-one positional relationship between the marker and the calibration feature by matching the image position information corresponding to each marker with the image position information of each calibration feature, and then sort the calibration features on the calibration part image to ensure the feasibility of calibrating the image capture device.

[0093] The above embodiments are merely illustrative of the principles and effects of this application and are not intended to limit this application. Anyone skilled in the art may modify or alter the above embodiments without departing from the spirit and scope of this application. Therefore, all equivalent modifications or alterations made by one of ordinary skill in the art without departing from the spirit and technical concepts disclosed in this application shall be covered by the claims of this application.

Claims

1. A method for sorting calibration features, characterized in that: The following steps are involved: Acquire a calibration part image, where the calibration part image is configured as an image obtained by photographing a calibration part configured with a plurality of markers, and the calibration part image includes a plurality of calibration features corresponding to the plurality of markers; identifying a plurality of calibration features in the calibration part image to determine a feature outer contour, the feature outer contour being configured to correspond to an outer contour of the plurality of calibration features; Based on the conversion relationship between the feature outer contour and a reference contour, the image position information corresponding to each marker is determined to sort the multiple calibration features; wherein the reference contour is configured to correspond to the outer contours of multiple markers.

2. The method for sorting calibration features according to claim 1, wherein: The calibration piece is configured to calibrate an internal parameter and / or an external parameter of an image capture device used to capture the calibration piece.

3. The method for sorting calibration features according to claim 2, wherein: The image capturing device is configured as an X-ray machine used in a surgical navigation system.

4. The method for sorting calibration features according to claim 1, wherein: The calibration component is configured as a calibration plate.

5. The method for sorting calibration features according to claim 1, wherein: The marker is configured to form a calibration point on the calibration piece.

6. The method for sorting calibration features according to claim 5, characterized in that: The calibration point is configured as a physical object or a pattern mark.

7. The method for sorting calibration features according to claim 1, wherein: The plurality of markers are configured to be distributed not on the same straight line so that the characteristic outer contour and the reference contour are polygonal.

8. The method for sorting calibration features according to claim 1, wherein: The step of identifying a plurality of calibration features in the calibration part image to determine the feature outer contour comprises: Performing feature recognition on the calibration part image to determine image position information of a plurality of calibration features; Based on the position information of the multiple calibration features, corner features are selected to form the feature outer contour.

9. The method for sorting calibration features according to claim 8, wherein: The selecting corner features based on the position information of the multiple calibration features to form the feature outer contour includes determining the feature outer contour using a convex hull algorithm.

10. The method for sorting calibration features according to claim 1, wherein: The step of determining the image position information corresponding to each marker based on the conversion relationship between the characteristic outer contour and a reference contour includes performing contour matching between the characteristic outer contour and the reference contour to determine the conversion relationship between the two.

11. The method for sorting calibration features according to claim 1, wherein: The step of determining the image position information corresponding to each marker based on the conversion relationship between the characteristic outer contour and a reference contour includes the step of determining the reference contour based on the physical position information of the multiple markers.

12. The method for sorting calibration features according to claim 1, wherein: The determining of the image position information corresponding to each marker based on the conversion relationship between the feature outer contour and a reference contour to sort the plurality of calibration features comprises the following steps: Acquire image position information corresponding to each marker based on the physical position information of the multiple markers and the conversion relationship; The image position information corresponding to each marker is matched with the image position information of each calibration feature to determine the calibration features corresponding to each marker, so as to sort the calibration features.

13. The method for sorting calibration features according to claim 11 or 12, characterized in that: The physical location information of the plurality of markers includes sequences and physical coordinates of the markers.

14. The method for sorting calibration features according to claim 11 or 12, characterized in that: The physical position information of the plurality of markers is determined based on the size information of the calibration plate and the size information between the markers.

15. A calibration method for an image pickup device, characterized in that: The following steps are involved: Sorting a plurality of calibration features on a calibration part image, wherein the sequence of the plurality of calibration features after sorting is consistent with the sequence of the plurality of markers configured on the calibration part; wherein the sorting method can be configured as the calibration feature sorting method according to any one of claims 1 to 14; The image capture device is calibrated with internal or external parameters based on the multiple calibration features and a preset calibration algorithm.

16. A device for sorting by calibrating features, characterized in that: include: An image acquisition module is configured to acquire a calibration part image, wherein the calibration part image is configured as an image obtained by photographing a calibration part configured with a plurality of markers, and the calibration part image includes a plurality of calibration features corresponding to the plurality of markers; a feature outer contour determining module, configured to identify the plurality of calibration features based on the calibration part image to determine a feature outer contour, wherein the feature outer contour is configured to correspond to an outer contour of the plurality of calibration features; A sorting module is used to determine the image position information corresponding to each marker based on the conversion relationship between the feature outer contour and a reference contour, so as to sort the multiple calibration features based on the image position data corresponding to each marker; wherein the reference contour is configured to correspond to the outer contours of multiple markers.

17. A calibration device for an image pickup device, characterized in that: include: a sorting module configured as the sorting device according to claim 16; A calibration module is configured to perform internal or external calibration on the image capture device based on the multiple calibration features involved in the sorting module and a preset calibration algorithm.

18. A computer device, characterized in that: include: a storage device for storing at least one program; A processing device is connected to the storage device, and is used to call the at least one program from the storage device and implement the sorting method as described in any one of claims 1 to 14, or the calibration method as described in claim 15 when executing it.

19. A computer-readable storage medium, characterized in that At least one program is stored, and when the at least one program is called and executed by a processor of a computer, the sorting method according to any one of claims 1 to 14 or the calibration method according to claim 15 is implemented.

20. A computer program product, characterized in that When the computer program product is run on a computer, the computer is enabled to execute the sorting method according to any one of claims 1 to 14 or the calibration method according to claim 15 .

Images