Method and device for registering jaw images, computer device and readable storage medium

Abstract

The application discloses a jaw image registration method and device, computer equipment and a readable storage medium, and relates to the field of image processing. The method comprises the following steps: determining a metal region in a jaw image; performing connected domain analysis on the metal region to obtain a plurality of metal sub-regions; generating a three-dimensional oriented bounding box of each metal sub-region; determining a target bounding box with a similar size to the size of a target two-dimensional oriented bounding box; determining the geometric center of the target bounding box and the target gravity center of the metal strip surrounded by the target bounding box; constructing a positioning plate coordinate system according to the target bounding box, the geometric center and the target gravity center; determining the first conversion relationship between the positioning plate coordinate system and the image coordinate system according to the geometric center and the target gravity center; and completing the jaw image registration based on the first conversion relationship. Thus, the positioning reference object in the positioning plate can be directly identified, the noise of other metal implants in the oral cavity is eliminated, and manual selection of the positioning reference object is not required.

Description

Technical Field

[0001] This invention relates to the field of image processing, and more particularly to a method, apparatus, computer device, and readable storage medium for registering jaw images. Background Technology

[0002] In dental image-guided surgery, it is often necessary to register the patient's CBCT (cone beam computer tomography) images of the jaw so that the doctor can clearly understand the condition inside the patient's mouth.

[0003] In a common registration method, the patient's teeth are fixedly connected to a registration block containing a positioning reference object and an optical reference array. Then, in the CBCT image, coordinate system transformation and registration are completed through the relationship between the positioning reference object, the optical reference array, the camera viewpoint, and the image coordinates. The positioning reference object is typically multiple metallic spheres.

[0004] Understandably, this method requires manual identification of the positioning reference object. However, there may be other metal implants in the patient's jaw, which may cause interference and make it difficult to accurately identify the positioning reference object, resulting in low registration efficiency and accuracy. Summary of the Invention

[0005] In view of this, the present invention provides a method, apparatus, computer device and readable storage medium for registering jaw images, so as to improve the current situation where the positioning reference object is difficult to be accurately identified, resulting in low efficiency and accuracy of registration.

[0006] In a first aspect, embodiments of the present invention provide a method for registering a jaw image. The jaw image is obtained by taking a picture of the user's jaw using a cone-beam computed reconstructive tomography (CBCT) device after placing a registration block in the user's oral cavity. The registration block includes a positioning plate with a metal strip on it. The metal strip forms an open planar shape, and the axis of symmetry of the target two-dimensional oriented bounding box corresponding to the planar shape does not coincide with the centroid of the metal strip. The size of the two-dimensional oriented bounding box is larger than the size of the two-dimensional oriented bounding box corresponding to other metal objects in the user's oral cavity. The method includes:

[0007] Based on a preset grayscale threshold, the metal region in the jaw image is determined, wherein the metal region is located in the image coordinate system;

[0008] Perform connectivity analysis on the metal region to obtain non-connected metal sub-regions within the metal region;

[0009] Generate a three-dimensional oriented bounding box corresponding to each of the said metal sub-regions;

[0010] Among all the three-dimensional directed bounding boxes, there exists a target bounding box whose face size is similar to that of the target two-dimensional directed bounding box;

[0011] Determine the geometric center of the target bounding box, and determine the target centroid of the metal strip enclosed by the target bounding box;

[0012] Construct a positioning plate coordinate system based on the target bounding box, the geometric center, and the target centroid;

[0013] Based on the geometric center and the target centroid, a first transformation relationship between the positioning plate coordinate system and the image coordinate system is determined;

[0014] Based on the first conversion relationship, the registration of the jaw image is completed.

[0015] Optionally, in one feasible embodiment of the present invention, determining that among all the three-dimensional directed bounding boxes there exists a target bounding box whose face size is similar to that of the target two-dimensional directed bounding box includes:

[0016] For each of the three-dimensional directed bounding boxes, determine whether there is a first parameter whose difference from the length of the target two-dimensional directed bounding box is less than a first preset value, and a second parameter whose difference from the width of the target two-dimensional directed bounding box is less than a second preset value, among the three parameters of the length, width and height of the three-dimensional directed bounding box.

[0017] If it exists, the three-dimensional oriented bounding box to which the first parameter and the second parameter belong is taken as the target bounding box.

[0018] Optionally, in one feasible embodiment of the present invention, the method further includes:

[0019] If it does not exist, a detection failure message is generated and sent to the staff's terminal.

[0020] Optionally, in one feasible embodiment of the present invention, constructing a positioning plate coordinate system based on the target bounding box, the geometric center, and the target centroid includes:

[0021] The geometric center is used as the origin of the coordinate system of the positioning plate to be constructed. An X-axis and a Y-axis are generated based on the origin. The X-axis is parallel to the axis corresponding to the smaller value of the first parameter or the second parameter. The Y-axis is parallel to the axis corresponding to the larger value of the first parameter or the second parameter. The target centroid of the metal strip is located in the region formed by the positive half-axis of the X-axis and the positive half-axis of the Y-axis.

[0022] The Z-axis is generated based on the cross product of the X-axis and the Y-axis, thus obtaining the coordinate system of the positioning plate.

[0023] Optionally, in one feasible embodiment of the present invention, determining the first transformation relationship between the positioning plate coordinate system and the image coordinate system based on the geometric center and the target centroid includes:

[0024] Calculate the pointing vector from the geometric center to the target centroid;

[0025] Based on the pointing vector, the coordinate axes of the positioning plate coordinate system are determined in the first direction of the image coordinate system, wherein the direction description includes an X-axis description, a Y-axis description, and a Z-axis description, and the Z-axis description is the cross product of the X-axis description and the Y-axis description;

[0026] Based on the X-axis description, the Y-axis description, the Z-axis description, and the geometric center, a first transformation relationship between the positioning plate coordinate system and the image coordinate system is determined.

[0027] Optionally, in one feasible embodiment of the present invention, the X-axis description and the Y-axis description respectively include:

[0028]

[0029]

[0030] In the formula, and The X-axis and Y-axis descriptions are respectively represented, a' and b' represent the length vector and width vector of the target bounding box, respectively, and k represents the pointing vector.

[0031] Optionally, in one feasible embodiment of the present invention, the registration block further includes an optical reference component connected to the positioning plate, the optical reference component including a plurality of reflective spheres, and the method further includes:

[0032] Obtain the engineering parameters of the registration block, wherein the engineering parameters are used to describe the first position of the plurality of reflective spheres and the second position of the metal strip in the positioning plate;

[0033] An optical reference coordinate system is constructed based on the first position of the plurality of reflective spheres;

[0034] Based on the second position of the metal strip, the centroid of the metal strip is determined to be at a third position in the optical reference coordinate system, and the target two-dimensional oriented bounding box of the metal strip is determined to be at a fourth position in the optical reference coordinate system.

[0035] Based on the third and fourth positions, the coordinate axis direction description and origin position description of the positioning plate coordinate system in the optical reference coordinate system are constructed.

[0036] Based on the coordinate axis direction description and the origin position description, a second transformation relationship between the positioning plate coordinate system and the optical reference coordinate system is determined.

[0037] The registration of the jaw image based on the first transformation relationship includes:

[0038] Based on the first transformation relationship, the registration of the jaw image is completed.

[0039] Secondly, embodiments of the present invention provide a registration device for a jaw image. The jaw image is obtained by taking a picture of the user's jaw using a cone-beam computed tomography (CBCT) device after placing a registration block in the user's oral cavity. The registration block includes a positioning plate with a metal strip on it. The metal strip forms an open planar shape, and the axis of symmetry of the target two-dimensional oriented bounding box corresponding to the planar shape does not coincide with the centroid of the metal strip. The size of the two-dimensional oriented bounding box is larger than the size of the two-dimensional oriented bounding box corresponding to other metal objects in the user's oral cavity. The device includes:

[0040] A region determination module is used to determine the metal region in the jaw image based on a preset grayscale threshold, wherein the metal region is located in the image coordinate system;

[0041] The sub-region determination module is used to perform connectivity analysis on the metal region to obtain non-connected metal sub-regions within the metal region.

[0042] A generation module is used to generate a three-dimensional oriented bounding box corresponding to each of the metal sub-regions;

[0043] The bounding box determination module is used to determine, among all the three-dimensional directed bounding boxes, a target bounding box whose face size is similar to that of the target two-dimensional directed bounding box;

[0044] A center determination module is used to determine the geometric center of the target bounding box and the target centroid of the metal strip enclosed by the target bounding box.

[0045] The construction module is used to construct a positioning plate coordinate system based on the target bounding box, the geometric center, and the target centroid;

[0046] The relationship determination module is used to determine a first transformation relationship between the positioning plate coordinate system and the image coordinate system based on the geometric center and the target centroid;

[0047] The registration module is used to complete the registration of the jaw image based on the first transformation relationship.

[0048] Thirdly, embodiments of the present invention provide a computer device, including a memory and a processor, wherein the memory stores a computer program, and the computer program, when run on the processor, executes a registration method for jaw images as disclosed in any of the first aspects.

[0049] Fourthly, embodiments of the present invention provide a computer-readable storage medium storing a computer program that, when run on a processor, executes a registration method for jaw images as disclosed in any of the first aspects.

[0050] In the jaw image registration method provided in this embodiment of the invention, firstly, based on a grayscale threshold that enables metal to be visualized, the metal region in the jaw image is determined; then, connected component analysis is performed on the metal region to divide the complete metal region into multiple non-connected metal sub-regions; subsequently, a corresponding three-dimensional directed bounding box is generated for each metal sub-region; then, based on the dimensions of each face of each three-dimensional directed bounding box, a target bounding box is determined among all three-dimensional directed bounding boxes, whose face dimensions are similar to a pre-determined target two-dimensional directed bounding box; next, the geometric center of the target bounding box is determined, and the target centroid of the metal strip enclosed by the target bounding box is determined; then, a positioning plate coordinate system is constructed according to the positional relationship between the target bounding box, the geometric center, and the target centroid; based on the positions of the geometric center and the target centroid in the positioning plate coordinate system and in the image coordinate system, a first transformation relationship between the positioning plate coordinate system and the image coordinate system is determined; finally, based on the first transformation relationship, the registration of the jaw image is completed.

[0051] Based on this, the embodiments of the present invention, through a unique positioning plate setting, enable the positioning reference object in the positioning plate, i.e., the metal strip, to be directly identified by the computer device, thereby reducing the negative impact caused by other metal implants in the user's oral cavity in the prior art, and eliminating the need for manual selection / identification of the positioning reference object, thus significantly improving registration efficiency and registration accuracy. Attached Figure Description

[0052] To more clearly illustrate the technical solution of the present invention, the accompanying drawings used in the embodiments will be briefly described below. It should be understood that the following drawings only show some embodiments of the present invention and should not be regarded as a limitation on the scope of protection of the present invention. In the various drawings, similar components are numbered similarly.

[0053] Figure 1 A flowchart illustrating the registration method for jaw images provided in an embodiment of the present invention is shown.

[0054] Figure 2A schematic diagram of the positioning plate provided in an embodiment of the present invention is shown;

[0055] Figure 3 A schematic diagram of the positioning plate coordinate system provided in an embodiment of the present invention is shown;

[0056] Figure 4 A schematic diagram of the structure of the registration device for jaw images provided in an embodiment of the present invention is shown. Detailed Implementation

[0057] The technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments.

[0058] The components of the embodiments of the invention described and illustrated herein can typically be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the invention provided in the accompanying drawings is not intended to limit the scope of the claimed invention, but merely to illustrate selected embodiments of the invention. All other embodiments obtained by those skilled in the art based on the embodiments of the invention without inventive effort are within the scope of protection of the invention.

[0059] In the following, the terms “comprising,” “having,” and their cognates, which may be used in various embodiments of the invention, are intended only to indicate a particular feature, number, step, operation, element, component, or combination thereof, and should not be construed as excluding, firstly, the presence of one or more other features, numbers, steps, operations, elements, components, or combinations thereof, or adding the possibility of one or more features, numbers, steps, operations, elements, components, or combinations thereof.

[0060] Furthermore, the terms "first," "second," and "third" are used only to distinguish descriptions and should not be interpreted as indicating or implying relative importance.

[0061] Unless otherwise specified, all terms used herein (including technical and scientific terms) shall have the same meaning as commonly understood by one of ordinary skill in the art to which the various embodiments of the invention pertain. Terms (such as those defined in commonly used dictionaries) shall be interpreted as having the same meaning as in their contextual meaning in the relevant technical field and shall not be interpreted as having an idealized or overly formal meaning, unless clearly defined in the various embodiments of the invention.

[0062] Example 1

[0063] Reference Figure 1This diagram illustrates a flowchart of a method for registering jaw images according to an embodiment of the present invention. In this embodiment, the jaw image is obtained by taking a picture of the user's jaw using a cone-beam computed reconstructive tomography (CBCT) device after placing the registration block in the user's oral cavity. The registration block includes a positioning plate with a metal strip on it. The metal strip forms an open planar shape. The axis of symmetry of the target two-dimensional oriented bounding box corresponding to the planar shape does not coincide with the centroid of the metal strip. The size of the two-dimensional oriented bounding box is larger than the size of the two-dimensional oriented bounding box corresponding to other metal objects in the user's oral cavity.

[0064] It should be understood that the jaw images in this embodiment of the invention are obtained by scanning / capturing the user's jaw using a cone beam computed tomography (CBCT) device.

[0065] It should also be understood that before the cone-beam computed tomography (CBCT) device can photograph the user's jaw, the registration block provided in this embodiment of the invention needs to be placed inside the user's oral cavity so that the CBCT device can simultaneously photograph the user's jaw and the registration block in the jaw.

[0066] Furthermore, the registration block provided in this embodiment of the invention differs from the registration block in the prior art mainly in the positioning plate. Specifically, the registration block in the prior art may be inlaid with three or more metal balls as positioning references. After the jaw image is captured, the computer device identifies the position of the positioning reference in the jaw image and completes the registration of the jaw image based on the position of the positioning reference.

[0067] In this embodiment of the invention, the positioning plate is provided with a strip-shaped object made of metal, i.e., a metal strip. The metal strip forms an open planar shape. Furthermore, it must be ensured that the center of gravity of the metal strip does not fall on any axis of symmetry of the two-dimensional oriented bounding box of the two metal strips.

[0068] To better illustrate the positioning plate provided in the embodiments of the present invention, please refer to... Figure 2 The diagram illustrates a positioning plate provided in an embodiment of the present invention. The positioning plate 200 is provided with a metal strip 201, which forms an open planar shape. The center of gravity of the metal strip 201 is 202, and the corresponding two-dimensional oriented bounding box is... Figure 2 The dashed box in the image. It is not hard to see that none of the axes of symmetry of the dashed box coincides with the centroid 202.

[0069] It needs to be clarified that, Figure 2The positioning plate 200 shown is only one possible configuration, and the shape of the metal strip can be set according to the actual situation. In practical applications, the metal strip only needs to meet the aforementioned requirements, and the embodiments of the present invention do not limit the shape of the metal strip.

[0070] Furthermore, it should be noted that, since the embodiments of the present invention need to identify the metal strip mounted on the positioning plate in the jaw image, when there are other metal objects such as dental implants inside the user's oral cavity, it is necessary to ensure that the size of the two-dimensional oriented bounding box of the metal strip is larger than the size of the two-dimensional oriented bounding box of the other metal objects, so as to ensure that the positioning plate can be correctly identified.

[0071] Based on this, the registration method for jaw images provided in this embodiment of the invention includes:

[0072] Step S110: Based on a preset grayscale threshold, determine the metal region in the jaw image, wherein the metal region is located in the image coordinate system.

[0073] That is, in the embodiments of the present invention, the gray value that enables the metal to be visualized in the jaw image is used as the gray value threshold to highlight the metal in the jaw image in order to obtain the metal region, that is, to reconstruct all metal regions in the jaw image.

[0074] Optionally, in a preferred embodiment of the present invention, the grayscale threshold is the lowest grayscale value that enables the metal to be developed.

[0075] Optionally, in one feasible approach, embodiments of the present invention will determine the metallic region in the jaw image based on the vtkMarchingCubes method in the VTK function library.

[0076] Step S120: Perform connectivity analysis on the metal region to obtain non-connected metal sub-regions within the metal region.

[0077] In other words, the embodiments of the present invention will perform connected component analysis on the identified entire metal region to determine the non-connected sub-regions, i.e., metal sub-regions, within the entire metal region.

[0078] Optionally, in one feasible approach, embodiments of the present invention perform connected component analysis based on the vtkConnectivityFilter method in the VTK function library.

[0079] Step S130: Generate a three-dimensional oriented bounding box corresponding to each of the metal sub-regions.

[0080] That is, for each metal sub-region, the embodiments of the present invention will establish a three-dimensional oriented bounding box to determine the position of each metal sub-region in the jaw image.

[0081] It is understandable that since any metal object, such as a metal strip, within the user's jaw is a three-dimensional object, any metal sub-region encompasses a three-dimensional space, that is, it is encompassed by a three-dimensional oriented bounding box.

[0082] Furthermore, it can be understood that, since the embodiments of the present invention use metal strips, the length and width of the three-dimensional directed bounding box corresponding to the planar shape formed by the metal strips should both be greater than the height. That is, among the three-dimensional parameters of the three-dimensional directed bounding box corresponding to the metal strip, one parameter should be much smaller than the other two parameters. Therefore, the embodiments of the present invention will determine the three-dimensional directed bounding box of the metal strip based on the three-dimensional properties of the three-dimensional directed bounding box corresponding to each metal sub-region.

[0083] Furthermore, it is easy to understand that, in this embodiment of the invention, the size of the two-dimensional directed bounding box of the metal strip is larger than the size of the two-dimensional directed bounding box of other metal objects in the user's oral cavity. Therefore, in the three-dimensional directed bounding box corresponding to the metal sub-region, it can be known that any three-dimensional attribute x, y, and z that is not a three-dimensional directed bounding box corresponding to the metal strip should satisfy the following constraints:

[0084] a*b>x*y,x≥y≥z

[0085] a≠x or b≠y

[0086] Where a and b are the length and width of the two-dimensional oriented bounding box of the metal strip, respectively.

[0087] Step S140: Determine that among all the three-dimensional directed bounding boxes, there exists a target bounding box whose face size is similar to that of the target two-dimensional directed bounding box.

[0088] That is, in this embodiment of the invention, a bounding box similar to the two-dimensional directed bounding box of the metal strip will be searched from the three-dimensional directed bounding boxes of all metal sub-regions.

[0089] Understandably, the three-dimensional and two-dimensional directed bounding boxes of the metal strip should be identical in length and width. Therefore, the three-dimensional directed bounding box of the metal strip should have a face whose dimensions are identical or similar to those of the two-dimensional directed bounding box.

[0090] Optionally, in one feasible embodiment of the present invention, step S140 specifically includes:

[0091] For each of the three-dimensional directed bounding boxes, determine whether there is a first parameter whose difference from the length of the target two-dimensional directed bounding box is less than a first preset value, and a second parameter whose difference from the width of the target two-dimensional directed bounding box is less than a second preset value, among the three parameters of the length, width and height of the three-dimensional directed bounding box.

[0092] If it exists, the three-dimensional oriented bounding box to which the first parameter and the second parameter belong is taken as the target bounding box.

[0093] In other words, in actual working environments, the three-dimensional properties of the three-dimensional oriented bounding box of any metal object may not match the length, width and height of the metal object in the front view. For example, in the three-dimensional properties x, y and z of the three-dimensional oriented bounding box, x may be the length in the front view, or y or z may be the length in the front view.

[0094] Therefore, in this embodiment of the invention, the three-dimensional attribute parameters of the three-dimensional directed bounding box are compared one by one to determine whether there is a first parameter whose difference from the length of the target two-dimensional directed bounding box is less than a first preset value, and a second parameter whose difference from the width of the target two-dimensional directed bounding box is less than a second preset value.

[0095] It is conceivable that the length and width of the three-dimensional oriented bounding box of the metal strip may be greater than or less than the length and width of the two-dimensional oriented bounding box of the metal strip. Therefore, in this embodiment of the invention, it is actually determined whether there is an absolute value of the difference between the length / width of the target two-dimensional oriented bounding box and the three attribute parameters that is less than the first preset value / second preset value.

[0096] As an example, let the length and width of the target two-dimensional directed bounding box be a and b, respectively, and let the set of vectors corresponding to the length, width, and height of the three-dimensional directed bounding box be {l, m, n}. Repeat the following loop steps: (1) Take two vectors from {l, m, n} and denote them as a' and b'; (2) Determine whether the absolute value of the difference between a and |a'| is less than a first preset value, and determine whether the absolute value of the difference between b and |b'| is less than a second preset value; (3) If both are satisfied, take this three-dimensional directed bounding box as the target bounding box and end the loop; (4) If both cannot be satisfied at the same time, execute step (1) again until any combination of two parameters in {l, m, n} has been tried, then end the loop and jump to the loop steps of the next three-dimensional directed bounding box.

[0097] To better illustrate step (2) above, the corresponding formula for step (2) is given:

[0098] |a-|a′|| <argu1

[0099] |b-|b′|| <argu2

[0100] In the formula, argu1 and argu2 represent the first preset value and the second preset value, respectively.

[0101] In addition, it is conceivable that the first preset value and the second preset value mentioned above can be set according to the actual situation. For example, in a feasible method provided by the embodiment of the present invention, the first preset value and the second preset value are both 0.3, that is, the above argu1 and argu2 are both 0.3.

[0102] It is also conceivable that if the target bounding box cannot be found, that is, if any three-dimensional oriented bounding box does not have the first and second parameters mentioned above, it indicates that an unexpected error has occurred during the execution process, or that the shooting angle setting of the cone beam computed tomography equipment is inappropriate, and / or the placement angle of the registration block in the user's oral cavity is inappropriate, resulting in the failure to identify the target bounding box. Therefore, in one feasible way, the embodiment of the present invention will terminate this registration process.

[0103] In another feasible embodiment of the present invention, in addition to terminating the current registration process, the present invention will also notify the staff. That is, in one feasible embodiment, the method further includes:

[0104] If it does not exist, a detection failure message is generated and sent to the staff's terminal.

[0105] Step S150: Determine the geometric center of the target bounding box and the target centroid of the metal strip enclosed by the target bounding box.

[0106] It is understood that the geometric center of the target bounding box and the center of gravity of the metal strip are both located on the plane where the positioning plate is located, but they do not coincide. Therefore, in this embodiment of the invention, the plane where the positioning plate is located, that is, the plane where the metal strip is located, will be identified based on the geometric center of the target bounding box and the center of gravity of the metal strip.

[0107] Optionally, in one feasible approach, embodiments of the present invention will obtain the geometric center of the target bounding box based on the vtkOBBTree class in the VTK library, and obtain the target centroid of the metal strip enclosed by the target bounding box based on the vtkCenterOfMass method in the VTK library.

[0108] Step S160: Construct a positioning plate coordinate system based on the target bounding box, the geometric center, and the target centroid.

[0109] It is understandable that within the three-dimensional target bounding box of the metal strip, there exists a face that is similar to or identical to the two-dimensional target bounding box of the metal strip, and this face lies on the same plane as its two-dimensional target bounding box.

[0110] Furthermore, since this embodiment of the invention uses a metal strip, and the diameter of the metal strip is negligible, meaning the height of the three-dimensional target bounding box is negligible, the geometric center of the three-dimensional target bounding box is on the same plane as the two-dimensional target bounding box. Also, the center of gravity of the metal strip is on the same plane as the two-dimensional target bounding box.

[0111] Based on this, embodiments of the present invention generate corresponding positioning plate coordinate systems based on multiple points located on the same plane.

[0112] Step S170: Based on the geometric center and the target centroid, determine the first transformation relationship between the positioning plate coordinate system and the image coordinate system.

[0113] That is, in this embodiment of the invention, the positions of the geometric center and the target centroid in the positioning plate coordinate system and the positions of the geometric center and the target centroid in the image coordinate system are used to determine the transformation relationship between the two coordinate systems, namely the first transformation relationship.

[0114] Understandably, due to the different axis directions and origin settings of the positioning plate coordinate system, the transformation relationship between the image coordinate system and the positioning plate coordinate system can be set according to the actual situation.

[0115] Step S180: Based on the first conversion relationship, complete the registration of the jaw image.

[0116] That is, in this embodiment of the invention, the first transformation relationship between the positioning plate coordinate system and the image coordinate system is used to determine the position of each region / target / object in the jaw area in different coordinate systems, thereby realizing the real-time spatial positioning of the jaw area, that is, completing the registration.

[0117] It is understandable that the spatial positioning of the camera coordinate system can be achieved using existing technologies related to optical reference arrays.

[0118] Based on this, the embodiments of the present invention, through a unique positioning plate setting, enable the positioning reference object in the positioning plate, i.e., the metal strip, to be directly identified by the computer device, thereby reducing the negative impact caused by other metal implants in the user's oral cavity in the prior art, and eliminating the need for manual selection / identification of the positioning reference object, thus significantly improving registration efficiency and registration accuracy.

[0119] Optionally, step S160, in a feasible embodiment of the present invention, specifically includes:

[0120] The geometric center is used as the origin of the coordinate system of the positioning plate to be constructed. An X-axis and a Y-axis are generated based on the origin. The X-axis is parallel to the axis corresponding to the smaller value of the first parameter or the second parameter. The Y-axis is parallel to the axis corresponding to the larger value of the first parameter or the second parameter. The target centroid of the metal strip is located in the region formed by the positive half-axis of the X-axis and the positive half-axis of the Y-axis.

[0121] The Z-axis is generated based on the cross product of the X-axis and the Y-axis, thus obtaining the coordinate system of the positioning plate.

[0122] To better illustrate this feasible approach, please refer to [link / reference]. Figure 3 This diagram illustrates the coordinate system of the positioning plate provided in an embodiment of the present invention. It should be noted that... Figure 3 In this context, o represents the origin, and g represents the target centroid.

[0123] Understandably, in this feasible approach, the embodiment of the present invention uses the geometric center of the target bounding box as the origin of the positioning plate coordinate system, and the axis corresponding to the smaller value of the first or second parameter as the X-axis, i.e., the minor axis of the bounding box; and the axis corresponding to the smaller value of the first or second parameter, i.e., the major axis of the bounding box, as the Y-axis. Furthermore, in each quadrant / intersection region of the X and Y axes, the directions of the X and Y axes in the quadrant to which the target centroid g belongs are defined as positive directions.

[0124] Finally, since the Z-axis is perpendicular to the X-axis and the Y-axis and the structure formed by them, this embodiment of the invention obtains the Z-axis of the positioning plate coordinate system by calculating the cross product of the X-axis and the Y-axis, thereby completing the construction of the positioning plate coordinate system.

[0125] Optionally, step S170, in a feasible embodiment of the present invention, specifically includes:

[0126] Calculate the pointing vector from the geometric center to the target centroid;

[0127] Based on the pointing vector, the coordinate axes of the positioning plate coordinate system are determined in the first direction of the image coordinate system, wherein the direction description includes an X-axis description, a Y-axis description, and a Z-axis description, and the Z-axis description is the cross product of the X-axis description and the Y-axis description;

[0128] Based on the X-axis description, the Y-axis description, the Z-axis description, and the geometric center, a first transformation relationship between the positioning plate coordinate system and the image coordinate system is determined.

[0129] That is, based on the expression / description of the method vectors of each axis of the positioning plate coordinate system under the image coordinate system, the present invention determines the transformation relationship of a point / object in the positioning plate coordinate system and the image coordinate system, thus obtaining the first transformation relationship.

[0130] As an example, let the direction vector of the X-axis of the positioning plate coordinate system be expressed as (x'1, y'1, z'1) in the image coordinate system, the direction vector of the Y-axis be expressed as (x'2, y'2, z'2) in the image coordinate system, the direction vector of the Z-axis be expressed as (x'3, y'3, z'3) in the image coordinate system, and the origin be expressed as (x'0, y'0, z'0) in the image coordinate system. Then the first transformation relation T1 can be expressed as:

[0131]

[0132] It is understandable that, due to the different ways in which the axis directions and origin of the positioning plate coordinate system are set, the expression method described in the first aspect above will also be adjusted accordingly. For example, in one feasible embodiment of the present invention, the X-axis description and the Y-axis description respectively include:

[0133]

[0134]

[0135] In the formula, and The X-axis and Y-axis descriptions are respectively represented, a' and b' represent the length vector and width vector of the target bounding box, respectively, and k represents the pointing vector.

[0136] It is understandable that a' refers to the vector corresponding to the longer side of the target bounding box, and b' refers to the vector corresponding to the shorter side of the target bounding box.

[0137] It is also understandable that if the origin of the positioning plate coordinate system is the geometric center of the target bounding box, the X-axis of the positioning plate coordinate system is parallel to the axis corresponding to the smaller value of the first or second parameter mentioned above, the Y-axis is parallel to the axis corresponding to the larger value of the first or second parameter mentioned above, and the target centroid of the metal strip is located in the region formed by the positive half-axis of the X-axis and the positive half-axis of the Y-axis, then the absolute value of a' is the larger value of the first or second parameter mentioned above, and the absolute value of b' is the smaller value of the first or second parameter mentioned above.

[0138] Optionally, in one feasible embodiment of the present invention, the registration block further includes an optical reference component connected to the positioning plate, the optical reference component including a plurality of reflective spheres, and the method further includes:

[0139] Obtain the engineering parameters of the registration block, wherein the engineering parameters are used to describe the first position of the plurality of reflective spheres and the second position of the metal strip in the positioning plate;

[0140] An optical reference coordinate system is constructed based on the first position of the plurality of reflective spheres;

[0141] Based on the second position of the metal strip, the centroid of the metal strip is determined to be at a third position in the optical reference coordinate system, and the target two-dimensional oriented bounding box of the metal strip is determined to be at a fourth position in the optical reference coordinate system.

[0142] Based on the third and fourth positions, the coordinate axis direction description and origin position description of the positioning plate coordinate system in the optical reference coordinate system are constructed.

[0143] Based on the coordinate axis direction description and the origin position description, a second transformation relationship between the positioning plate coordinate system and the optical reference coordinate system is determined.

[0144] Therefore, regarding the above step S180, this optional method specifically includes:

[0145] Based on the first transformation relationship, the registration of the jaw image is completed.

[0146] It should be clarified that the engineering parameters in this embodiment refer to the parameters specified for the position and size of each component in the registration block during the product design stage. Based on this, this embodiment will read the position parameters of the reflective sphere and the metal strip according to the engineering parameters of the registration block, thereby determining the first position of the reflective sphere and the second position of the metal strip.

[0147] It is not difficult to understand that the first position and the second position in the embodiments of the present invention can actually be understood as the relative positional relationship between the reflective sphere and the metal strip. The embodiments of the present invention do not limit the coordinate system / observation space in which the first position and the second position are located, but only require that the first position and the second position can reflect the relative positional relationship between the reflective sphere and the metal strip.

[0148] It is also clearly understood that the optical reference component in the embodiments of the present invention refers to the optical reference array in the prior art. The optical reference component includes multiple reflective spheres, and the relative positions of each reflective sphere are preset / known. Therefore, after capturing an image of the jaw containing the reflective spheres, the transformation relationship between the camera coordinate system and the optical reference coordinate system can be determined based on the positions of each reflective sphere from the camera's perspective.

[0149] Additionally, it should be noted that the construction of the optical reference coordinate system can be set according to the actual situation. As an example, when there are three reflective spheres, the center of one of the reflective spheres is taken as the origin of the coordinate system. The line connecting any two of the centers of the three reflective spheres is taken as the first axis, and the normal to the plane formed by the centers of the three reflective spheres is taken as the second axis. The cross product of the first axis vector and the second axis vector is used to obtain the third axis, thus obtaining the optical reference coordinate system.

[0150] Therefore, after obtaining the optical reference coordinate system, the embodiments of the present invention will determine the centroid of the metal strip and the target two-dimensional oriented bounding box corresponding to the metal strip at the third and fourth positions in the optical reference coordinate system, respectively, based on the second position of the metal strip, that is, the position of the metal strip relative to the reflective sphere.

[0151] Next, based on the third position of the center of gravity and the fourth position of the target two-dimensional oriented bounding box, this embodiment of the invention will construct the expression / description of the positioning plate coordinate system corresponding to the metal strip in the optical reference coordinate system, that is, the coordinate axis direction description and the origin position description.

[0152] It should be noted that when constructing the coordinate axis direction description and origin position description of the positioning plate coordinate system under the optical reference coordinate system and the image coordinate system, the specifications of the positioning plate coordinate system under the optical reference coordinate system and the image coordinate system must be consistent. For example, in an embodiment of the present invention, the positioning plate coordinate system under the optical reference coordinate system and the image coordinate system has its X-axis parallel to the axis corresponding to the smaller value of the aforementioned first parameter or second parameter, and its Y-axis parallel to the axis corresponding to the larger value of the aforementioned first parameter or second parameter. Furthermore, the target centroid of the metal strip is located in the region formed by the positive half-axis of its X-axis and the positive half-axis of its Y-axis.

[0153] Finally, based on the directional descriptions of each coordinate axis of the positioning plate coordinate system corresponding to the metal strip in the optical reference coordinate system, and the position description of the origin of the positioning plate coordinate system in the optical reference coordinate system, the coordinate transformation relationship between the optical reference coordinate system and the positioning plate coordinate system is determined.

[0154] As an example, let the direction vector of the X-axis of the positioning plate coordinate system be expressed as (x1, y1, z1) in the optical reference coordinate system, the direction vector of the Y-axis be expressed as (x2, y2, z2) in the optical reference coordinate system, the direction vector of the Z-axis be expressed as (x3, y3, z3) in the optical reference coordinate system, and the origin be expressed as (x0, y0, z0) in the optical reference coordinate system. Then the second transformation relation T2 can be expressed as:

[0155]

[0156] Based on this, the transformation relationship T2 between the optical reference coordinate system and the image coordinate system can be expressed as:

[0157] T3 = T2·T1

[0158] Example 2

[0159] Corresponding to the registration method for jaw images provided in Embodiment 1 of the present invention, Embodiment 2 of the present invention also provides a registration device for jaw images, referring to... Figure 4 This diagram illustrates the structure of a registration device for jaw images provided in an embodiment of the present invention. In this embodiment, the jaw image is obtained by taking a picture of the user's jaw using a cone-beam computed tomography (CBCT) device after placing the registration block inside the user's oral cavity. The registration block includes a positioning plate with a metal strip on it. The metal strip forms an open planar shape, and the axis of symmetry of the target two-dimensional oriented bounding box corresponding to the planar shape does not coincide with the centroid of the metal strip. The size of the two-dimensional oriented bounding box is larger than the size of the two-dimensional oriented bounding boxes corresponding to other metal objects in the user's oral cavity. Based on this, the jaw image registration device 300 provided in this embodiment of the present invention includes:

[0160] The region determination module 310 is used to determine the metal region in the jaw image based on a preset grayscale threshold, wherein the metal region is located in the image coordinate system;

[0161] The sub-region determination module 320 is used to perform connectivity analysis on the metal region to obtain non-connected metal sub-regions in the metal region;

[0162] Generation module 330 is used to generate a three-dimensional oriented bounding box corresponding to each of the metal sub-regions;

[0163] Bounding box determination module 340 is used to determine, among all the three-dimensional directed bounding boxes, there is a target bounding box whose face size is similar to that of the target two-dimensional directed bounding box;

[0164] The center determination module 350 is used to determine the geometric center of the target bounding box and the target centroid of the metal strip enclosed by the target bounding box.

[0165] Construction module 360 ​​is used to construct a positioning plate coordinate system based on the target bounding box, the geometric center, and the target centroid;

[0166] The relationship determination module 370 is used to determine a first transformation relationship between the positioning plate coordinate system and the image coordinate system based on the geometric center and the target centroid.

[0167] The registration module 380 is used to complete the registration of the jaw image based on the first transformation relationship. Optionally, in one feasible embodiment of the present invention, the bounding box determination module includes:

[0168] The parameter determination submodule is used to determine, for each of the three parameters of the three-dimensional directed bounding box, whether there is a first parameter whose difference from the length of the target two-dimensional directed bounding box is less than a first preset value, and a second parameter whose difference from the width of the target two-dimensional directed bounding box is less than a second preset value.

[0169] The target bounding box determination submodule is used to, if it exists, take the three-dimensional oriented bounding box to which the first parameter and the second parameter belong as the target bounding box.

[0170] Optionally, in one feasible embodiment of the present invention, the apparatus further includes:

[0171] The sending module is used to generate a detection failure message if the function does not exist, and then send the detection failure message to the staff's terminal.

[0172] Optionally, in one feasible embodiment of the present invention, the construction module includes:

[0173] The first coordinate axis determination submodule is used to take the geometric center as the origin of the coordinate system of the positioning plate to be constructed, and generate the X-axis and Y-axis based on the origin. The X-axis is parallel to the axis corresponding to the smaller value of the first parameter or the second parameter, and the Y-axis is parallel to the axis corresponding to the larger value of the first parameter or the second parameter. The target centroid of the metal strip is located in the region formed by the positive half-axis of the X-axis and the positive half-axis of the Y-axis.

[0174] The second coordinate axis determination submodule is used to generate the Z axis based on the cross product of the X-axis and the Y-axis, thereby obtaining the positioning plate coordinate system.

[0175] Optionally, in one feasible embodiment of the present invention, the relationship determination module includes:

[0176] The vector calculation submodule is used to calculate the pointing vector from the geometric center to the target centroid;

[0177] The direction description determination submodule is used to determine the first direction description of the coordinate axes of the positioning plate coordinate system in the image coordinate system based on the pointing vector, wherein the direction description includes an X-axis description, a Y-axis description and a Z-axis description, and the Z-axis description is the cross product of the X-axis description and the Y-axis description;

[0178] The transformation relationship determination submodule is used to determine the first transformation relationship between the positioning plate coordinate system and the image coordinate system based on the X-axis description, the Y-axis description, the Z-axis description and the geometric center.

[0179] Optionally, in one feasible embodiment of the present invention, the X-axis description and the Y-axis description respectively include:

[0180]

[0181]

[0182] In the formula, and The X-axis and Y-axis descriptions are respectively represented, a' and b' represent the length vector and width vector of the target bounding box, respectively, and k represents the pointing vector.

[0183] Optionally, in one feasible embodiment of the present invention, the registration block further includes an optical reference component connected to the positioning plate, the optical reference component including a plurality of reflective spheres, and the device further includes:

[0184] An engineering parameter acquisition module is used to acquire the engineering parameters of the registration block, wherein the engineering parameters are used to describe the first position of the plurality of reflective spheres and the second position of the metal strip in the positioning plate;

[0185] An optical reference coordinate system construction module is used to construct an optical reference coordinate system based on the first position of the plurality of reflective spheres;

[0186] The position determination module is used to determine, based on the second position of the metal strip, the third position of the center of gravity of the metal strip in the optical reference coordinate system, and the fourth position of the target two-dimensional oriented bounding box of the metal strip in the optical reference coordinate system.

[0187] The position description construction module is used to construct the coordinate axis direction description and origin position description of the positioning plate coordinate system in the optical reference coordinate system based on the third position and the fourth position.

[0188] The second transformation relationship determination module is used to determine the second transformation relationship between the positioning plate coordinate system and the optical reference coordinate system based on the coordinate axis direction description and the origin position description.

[0189] The registration module is also used to complete the registration of the jaw image based on the first transformation relationship and the first transformation relationship.

[0190] The jaw image registration device 300 provided in this application embodiment can realize each process of the jaw image registration method corresponding to Embodiment 1, and can achieve the same technical effect. To avoid repetition, it will not be described again here.

[0191] This invention also provides a computer device, including a memory and a processor. The memory stores a computer program, and the computer program executes the registration method for jaw images as described in Embodiment 1 when it runs on the processor.

[0192] This invention also provides a computer-readable storage medium storing a computer program that, when run on a processor, executes the registration method for jaw images as described in Embodiment 1.

[0193] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can also be implemented in other ways. The apparatus embodiments described above are merely illustrative; for example, the flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of apparatus, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that, as an alternative implementation, the functions marked in the blocks may occur in a different order than those marked in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagram and / or flowchart, and combinations of blocks in the block diagram and / or flowchart, can be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

[0194] In addition, the functional modules or units in the various embodiments of the present invention can be integrated together to form an independent part, or each module can exist independently, or two or more modules can be integrated to form an independent part.

[0195] If the aforementioned functions are implemented as software functional modules and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a smartphone, personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0196] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for registering jaw images, characterized in that, The jaw image is obtained by taking a picture of the user's jaw using a cone-beam computed tomography (CBCT) device after placing a registration block in the user's oral cavity. The registration block includes a positioning plate with metal strips forming an open planar shape. The axis of symmetry of the target two-dimensional oriented bounding box corresponding to the planar shape does not coincide with the centroid of the metal strips. The size of the two-dimensional oriented bounding box is larger than the size of the two-dimensional oriented bounding boxes corresponding to other metal objects in the user's oral cavity. The method includes: Based on a preset grayscale threshold, the metal region in the jaw image is determined, wherein the metal region is located in the image coordinate system; Perform connectivity analysis on the metal region to obtain non-connected metal sub-regions within the metal region; Generate a three-dimensional oriented bounding box corresponding to each of the said metal sub-regions; Among all the three-dimensional directed bounding boxes, there exists a target bounding box whose face size is similar to that of the target two-dimensional directed bounding box; Determine the geometric center of the target bounding box, and determine the target centroid of the metal strip enclosed by the target bounding box; Construct a positioning plate coordinate system based on the target bounding box, the geometric center, and the target centroid; Based on the geometric center and the target centroid, a first transformation relationship between the positioning plate coordinate system and the image coordinate system is determined; Based on the first conversion relationship, the registration of the jaw image is completed.

2. The registration method for jaw images according to claim 1, characterized in that, The step of determining that among all the three-dimensional directed bounding boxes, there exists a target bounding box whose face size is similar to that of the target two-dimensional directed bounding box includes: For each of the three-dimensional directed bounding boxes, determine whether there is a first parameter whose difference from the length of the target two-dimensional directed bounding box is less than a first preset value, and a second parameter whose difference from the width of the target two-dimensional directed bounding box is less than a second preset value, among the three parameters of the length, width and height of the three-dimensional directed bounding box. If it exists, the three-dimensional oriented bounding box to which the first parameter and the second parameter belong is taken as the target bounding box.

3. The registration method for jaw images according to claim 2, characterized in that, The method further includes: If it does not exist, a detection failure message is generated and sent to the staff's terminal.

4. The registration method for jaw images according to claim 2, characterized in that, The step of constructing a positioning plate coordinate system based on the target bounding box, the geometric center, and the target centroid includes: The geometric center is used as the origin of the coordinate system of the positioning plate to be constructed. An X-axis and a Y-axis are generated based on the origin. The X-axis is parallel to the axis corresponding to the smaller value of the first parameter or the second parameter. The Y-axis is parallel to the axis corresponding to the larger value of the first parameter or the second parameter. The target centroid of the metal strip is located in the region formed by the positive half-axis of the X-axis and the positive half-axis of the Y-axis. The Z-axis is generated based on the cross product of the X-axis and the Y-axis, thus obtaining the coordinate system of the positioning plate.

5. The registration method for jaw images according to claim 1, characterized in that, Determining the first transformation relationship between the positioning plate coordinate system and the image coordinate system based on the geometric center and the target centroid includes: Calculate the pointing vector from the geometric center to the target centroid; Based on the pointing vector, the coordinate axes of the positioning plate coordinate system are determined in the first direction of the image coordinate system, wherein the direction description includes an X-axis description, a Y-axis description, and a Z-axis description, and the Z-axis description is the cross product of the X-axis description and the Y-axis description; Based on the X-axis description, the Y-axis description, the Z-axis description, and the geometric center, a first transformation relationship between the positioning plate coordinate system and the image coordinate system is determined.

6. The registration method for jaw images according to claim 5, characterized in that, The X-axis description and the Y-axis description each include: In the formula, and The X-axis and Y-axis descriptions are respectively represented, a' and b' represent the length vector and width vector of the target bounding box, respectively, and k represents the pointing vector.

7. The registration method for jaw images according to claim 1, characterized in that, The registration block further includes an optical reference assembly connected to the positioning plate, the optical reference assembly including a plurality of reflective spheres, and the method further includes: Obtain the engineering parameters of the registration block, wherein the engineering parameters are used to describe the first position of the plurality of reflective spheres and the second position of the metal strip in the positioning plate; An optical reference coordinate system is constructed based on the first position of the plurality of reflective spheres; Based on the second position of the metal strip, the centroid of the metal strip is determined to be at a third position in the optical reference coordinate system, and the target two-dimensional oriented bounding box of the metal strip is determined to be at a fourth position in the optical reference coordinate system. Based on the third and fourth positions, the coordinate axis direction description and origin position description of the positioning plate coordinate system in the optical reference coordinate system are constructed. Based on the coordinate axis direction description and the origin position description, a second transformation relationship between the positioning plate coordinate system and the optical reference coordinate system is determined. The registration of the jaw image based on the first transformation relationship includes: Based on the first transformation relationship, the registration of the jaw image is completed.

8. A registration device for jaw images, characterized in that, The jaw image is obtained by taking a picture of the user's jaw using a cone-beam computed tomography (CBCT) device after placing a registration block in the user's oral cavity. The registration block includes a positioning plate with metal strips forming an open planar shape. The axis of symmetry of the target two-dimensional oriented bounding box corresponding to the planar shape does not coincide with the centroid of the metal strips. The size of the two-dimensional oriented bounding box is larger than the size of the two-dimensional oriented bounding boxes corresponding to other metal objects in the user's oral cavity. The device includes: A region determination module is used to determine the metal region in the jaw image based on a preset grayscale threshold, wherein the metal region is located in the image coordinate system; The sub-region determination module is used to perform connectivity analysis on the metal region to obtain non-connected metal sub-regions within the metal region. A generation module is used to generate a three-dimensional oriented bounding box corresponding to each of the metal sub-regions; The bounding box determination module is used to determine, among all the three-dimensional directed bounding boxes, a target bounding box whose face size is similar to that of the target two-dimensional directed bounding box; A center determination module is used to determine the geometric center of the target bounding box and the target centroid of the metal strip enclosed by the target bounding box. The construction module is used to construct a positioning plate coordinate system based on the target bounding box, the geometric center, and the target centroid; The relationship determination module is used to determine a first transformation relationship between the positioning plate coordinate system and the image coordinate system based on the geometric center and the target centroid; The registration module is used to complete the registration of the jaw image based on the first transformation relationship.

9. A computer device, characterized in that, It includes a memory and a processor, the memory storing a computer program that, when run on the processor, executes the registration method for jaw images as described in any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when run on a processor, executes the registration method for jaw images as described in any one of claims 1-7.

Images