Oral image positioning navigation method and system

Abstract

The present application provides an oral image positioning and navigation method and system, wherein the oral image positioning and navigation method is applied to the positioning and navigation system of oral surgery equipment. The method includes: performing oral imaging of an oral cavity with a positioning device and a positioning element and generating oral imaging results; calculating the coordinates of the metal reflective device and the positioning element in the three-dimensional image space according to the oral imaging results, and forming a three-dimensional space coordinate set; calculating the coordinates of the metal reflective ball and the positioning element in the local coordinate system of the optical positioning system, and forming a local coordinate set of the optical positioning system; generating a transformation matrix; and obtaining the optical positioning coordinates of the optical positioning device in the local coordinate system of the optical positioning system; calculating the coordinates of the optical positioning device in the three-dimensional image space according to the optical positioning coordinates and the transformation matrix. The method and system provided by the embodiments of the present application can not only improve the accuracy of image positioning and navigation in oral surgery, but also improve the speed of positioning the surgical area.

Description

Technical Field

[0001] The present application relates to the field of image positioning, and in particular, to an oral image positioning and navigation method, system, electronic device, and computer-readable storage medium. Background Art

[0002] With the improvement of people's living standards and the aging of the population, the demand for dental surgery is increasing. Dental surgery equipment such as dental robots are usually used for surgery. The positioning speed and accuracy of the positioning navigation system of such equipment are crucial to dental surgery.

[0003] Currently, commonly used image positioning and navigation methods include positioning through full-mouth guides with metal pins and algorithms, or positioning through positioning guides with metal reflective balls and algorithms. However, these methods are often accompanied by problems such as long positioning time or poor accuracy, which affects the quality and speed of the surgery. Summary of the invention

[0004] The purpose of the embodiments of the present application is to provide an oral image positioning and navigation method and system, which performs oral imaging of an oral cavity with a positioning device and a positioning element and generates an oral imaging result; further, generates a three-dimensional space coordinate set including the coordinates of a metal reflective device and a positioning element in a three-dimensional space, and generates an optical positioning system local coordinate set including the coordinates of a metal reflective ball and a positioning element in an optical positioning system; generates a transformation matrix based on the three-dimensional space coordinate set and the learned positioning system local coordinate set; generates a three-dimensional image of the optical positioning device based on the transformation matrix and the optical positioning coordinates of the optical positioning device in the local coordinate system of the optical positioning system, and displays it in a navigation system for real-time navigation; using this method for positioning and navigation has high accuracy and fast positioning speed.

[0005] In a first aspect, an embodiment of the present application provides an oral image positioning and navigation method, characterized in that the method is applied to a positioning and navigation system of an oral surgery device to determine the surgical execution site; the method comprises: performing oral imaging of an oral cavity with a positioning device and a positioning element and generating an oral imaging result; wherein the positioning device comprises a metal reflective device; according to the oral imaging result, calculating the coordinates of the metal reflective device and the positioning element in a three-dimensional image space and forming a three-dimensional space coordinate set; calculating the coordinates of the metal reflective ball and the positioning element in a local coordinate system of an optical positioning system and forming a local coordinate set of the optical positioning system; generating a transformation matrix according to the three-dimensional space coordinate set and the local coordinate set of the optical positioning system; wherein the transformation matrix is ​​used to convert the local coordinate system of the optical positioning system into the three-dimensional space coordinate system; and obtaining the optical positioning coordinates of the optical positioning device in the local coordinate system of the optical positioning system; calculating the coordinates of the optical positioning device in the three-dimensional image space according to the optical positioning coordinates and the transformation matrix; wherein the optical positioning device is used to capture its position in the optical positioning system and guide its direction in the positioning and navigation system.

[0006] In the above implementation process, the oral image using the positioning device and the positioning element at the same time is processed to greatly improve the accuracy of image positioning; then the transformation matrix that converts the local coordinate set of the optical positioning system of the metal reflective device and the positioning element into a three-dimensional space coordinate set is calculated; this matrix represents the transformation relationship between the local coordinate system of the optical positioning system and the three-dimensional space coordinate system, and the transformation matrix and the optical positioning device are operated to obtain the image of the optical positioning device in the three-dimensional space. The whole process does not require manual collection or capture of coordinates, and has high efficiency and fast positioning speed.

[0007] Optionally, in an embodiment of the present application, the positioning element is installed in an area adjacent to the surgical execution area, and the positioning device is installed on the opposite side of the positioning element; the positioning device also includes a positioning guide and an infrared reflecting device for determining the coordinate system of the optical positioning system; the infrared reflecting device is installed on the positioning guide.

[0008] In the above implementation process, the positioning device is installed on the opposite side of the surgical execution area, and the positioning element is installed on the opposite side of the positioning device and in the adjacent area of ​​the surgical execution area. The use of the positioning device and the positioning element greatly improves the positioning accuracy of the method, so that the optical positioning device can be accurately positioned in the surgical execution area.

[0009] Optionally, in an embodiment of the present application, calculating the coordinates of the metal reflective device and the positioning element in the three-dimensional image space based on the oral imaging results includes: obtaining the offset of the metal reflective device and the positioning element from the coordinate origin in the three-dimensional space from the imaging results; the coordinate origin is determined by the infrared reflective device; converting the oral imaging results into a grayscale image; calculating the center coordinates of the metal reflective device and the positioning element on the grayscale image; and calculating the coordinates of the metal reflective device and the positioning element in the three-dimensional image space based on the image serial number of the imaging result, the offset and the center coordinates.

[0010] In the above implementation process, the coordinates of the metal reflective device and the positioning element in the three-dimensional image space are calculated according to the image serial number, offset and center coordinates of the imaging results. The serial number of each image is fixed, each offset is determined by its offset from the coordinate origin in the three-dimensional space, and the center coordinates are determined by a specific grayscale image. The coordinates of the metal reflective device and the positioning element in the three-dimensional image space calculated in this way are very accurate.

[0011] Optionally, in an embodiment of the present application, calculating the coordinates of the metal reflective device and the positioning element in the three-dimensional image space based on the picture serial number of the imaging result, the offset and the center coordinates includes: determining the horizontal coordinate in the three-dimensional image space as the sum of the product of the horizontal coordinate of the center coordinate and the pixel interval of the imaging result and the horizontal coordinate of the offset; determining the vertical coordinate in the three-dimensional image space as the sum of the product of the vertical coordinate of the center coordinate and the pixel interval of the imaging result and the vertical coordinate of the offset; determining the vertical coordinate in the three-dimensional image space as the sum of the product of the picture serial number of the imaging result and the pixel interval of the imaging result and the vertical coordinate of the offset.

[0012] In the above implementation process, the horizontal coordinate is determined according to the product of the horizontal coordinate of the center coordinate and the pixel spacing of the imaging result and the sum of the horizontal coordinate of the offset; the vertical coordinate is determined according to the product of the vertical coordinate of the center coordinate and the pixel spacing of the imaging result and the sum of the vertical coordinate of the offset; the vertical coordinate is determined according to the product of the image sequence number of the imaging result and the pixel spacing of the imaging result and the sum of the vertical coordinate of the offset. By determining the coordinate axes of the metal reflective device and the positioning element in the three-dimensional space in this way, the coordinates of the metal reflective device and the positioning element in the three-dimensional image space can be accurately and quickly obtained.

[0013] Optionally, in an embodiment of the present application, the calculation of the coordinates of the metal reflective ball and the positioning element in the local coordinate system of the optical positioning system includes: establishing the local coordinate system of the optical positioning system of the positioning device according to the positioning guide; obtaining the coordinates of the metal reflective device, the positioning element and the positioning guide in the optical positioning system; obtaining the coordinates of the metal reflective device in the local coordinate system of the optical positioning system according to the positioning device; obtaining the coordinates of the positioning element in the local coordinate system of the optical positioning system according to the coordinates of the positioning element in the optical positioning system and the coordinates of the positioning guide in the optical positioning system.

[0014] In the above implementation process, the coordinates of the metal reflective device in the local coordinate system of the optical positioning system are obtained according to the positioning device, and the coordinates of the positioning element in the local coordinate system of the optical positioning system are determined by the coordinates of the positioning element in the optical positioning system and the coordinates of the positioning guide in the optical positioning system. This method accurately obtains the coordinates of the positioning element in the local coordinate system of the optical positioning system, effectively increasing the positioning and identification accuracy of the surgical execution area.

[0015] Optionally, in an embodiment of the present application, establishing a local coordinate system of the optical positioning system of the positioning device based on the positioning guide includes: identifying the center of the infrared reflection device in the optical positioning system as the coordinate origin of the local coordinate system of the optical positioning system of the positioning device; the horizontal axis, longitudinal axis and vertical axis of the positioning device in the optical positioning system are consistent with a preset orientation of the positioning device.

[0016] In the above implementation process, the center of the infrared reflective ball is identified as the origin of the coordinate axis in the optical positioning system, and the horizontal axis, longitudinal axis and vertical axis are determined by the preset direction of the positioning device, so that the coordinate axis of the positioning device in the optical positioning system is consistent with its preset direction, ensuring that the image displayed on the navigation system is consistent with the actual situation in the oral cavity, improving the accuracy of positioning navigation, and ensuring the accurate performance of oral surgery.

[0017] Optionally, in an embodiment of the present application, after calculating the coordinates of the optical positioning device in the three-dimensional image space, the method further includes: transmitting and displaying the three-dimensional space image of the optical positioning device on the navigation system; the three-dimensional image guides the oral surgery equipment robotic arm to perform surgery.

[0018] In the above implementation process, the coordinates of the three-dimensional image space of the optical positioning device obtained in the method are transmitted to the navigation system, and the surgery is performed according to the display of the navigation system. The oral surgery is accurately guided according to the three-dimensional space image of the optical positioning device, making the surgery using the robotic arm more accurate and faster, and improving the patient's experience during the operation.

[0019] In a second aspect, an embodiment of the present application provides a positioning and navigation system, characterized in that the positioning and navigation system includes: an oral imaging module, which is used to perform oral imaging of an oral cavity with a positioning device and a positioning element and generate an oral imaging result; wherein the positioning device includes a metal reflective device; a three-dimensional space coordinate set generation module, which is used to calculate the coordinates of the metal reflective device and the positioning element in the three-dimensional image space according to the oral imaging result, and form a three-dimensional space coordinate set; an optical positioning system local coordinate set generation module, which is used to calculate the coordinates of the metal reflective ball and the positioning element in the local coordinate system of the optical positioning system, and form a local coordinate set of the optical positioning system; a transformation matrix generation module, which is used to generate a transformation matrix according to the three-dimensional space coordinate set and the local coordinate set of the optical positioning system; wherein the transformation matrix is ​​used to convert the local coordinate system of the optical positioning system into the three-dimensional space coordinate system; and an optical positioning device navigation module, which is used to obtain the optical positioning coordinates of the optical positioning device in the local coordinate system of the optical positioning system; calculate the coordinates of the optical positioning device in the three-dimensional image space according to the optical positioning coordinates and the transformation matrix; wherein the optical positioning device is used to capture its position in the optical positioning system and guide the direction in the positioning and navigation system.

[0020] In the above implementation process, the oral imaging module generates oral imaging results and calculates the three-dimensional space coordinate set; the optical positioning system local coordinate set generation module generates the optical positioning system local coordinate set; the transformation matrix generation module generates a transformation matrix for converting the local coordinate system of the optical positioning system into the three-dimensional space coordinate system; the optical positioning device navigation module calculates the coordinates of the optical positioning device in the three-dimensional image space, and guides the operation in the navigation system. The image in the navigation system corresponds to the actual oral situation, avoiding the situation where the operation is affected by the inaccurate positioning of the positioning navigation system.

[0021] In a third aspect, an embodiment of the present application provides an electronic device, comprising a memory and a processor, wherein the memory stores program instructions, and when the processor reads and runs the program instructions, it executes the steps in any of the above implementation methods.

[0022] In a fourth aspect, an embodiment of the present application further provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer program instructions, and when the computer program instructions are read and executed by a processor, the steps in any of the above-mentioned implementation methods are executed. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments of the present application will be briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0024] Figure 1 A flowchart of a first oral image positioning and navigation method provided in an embodiment of the present application;

[0025] Figure 2 A position relationship diagram in an application scenario of the oral image positioning and navigation method provided in an embodiment of the present application;

[0026] Figure 3 A flowchart of a second oral image positioning and navigation method provided in an embodiment of the present application;

[0027] Figure 4 A flowchart of a third oral image positioning and navigation method provided in an embodiment of the present application;

[0028] Figure 5 A flowchart of a fourth oral image positioning and navigation method provided in an embodiment of the present application;

[0029] Figure 6 A schematic diagram of a local coordinate system of an optical positioning system of an oral image positioning and navigation method provided in an embodiment of the present application;

[0030] Figure 7 A schematic diagram of the oral image positioning and navigation system module provided in an embodiment of the present application; and

[0031] Figure 8 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0032] The technical solutions in the embodiments of the present application will be described below in conjunction with the accompanying drawings in the embodiments of the present application. For example, the flowcharts and block diagrams in the accompanying drawings show the possible architectures, functions and operations of the systems, methods and computer program products according to multiple embodiments of the present invention. In this regard, each box in the flowchart or block diagram may represent a module, a program segment or a part of a code, and the module, a program segment or a part of a code contains one or more executable instructions for implementing the specified logical functions. It should also be noted that in some alternative implementations, the functions marked in the box may also occur in an order different from that marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they may sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or the flowchart, and the combination of boxes in the block diagram and / or the flowchart, can be implemented by a dedicated hardware-based system that performs a specified function or action, or can be implemented by a combination of dedicated hardware and computer instructions. In addition, the functional modules in the various embodiments of the present invention can be integrated together to form an independent part, or each module can exist separately, or two or more modules can be integrated to form an independent part.

[0033] During the research process, the applicant found that the positioning and navigation methods of the positioning and navigation systems currently used in oral surgery equipment include positioning by applying an algorithm through a full-mouth guide with metal pins, or positioning by applying an algorithm using a positioning guide with metal reflective balls. These two methods have the problems of slow image production and poor accuracy, which often affect the progress of oral surgery.

[0034] Based on this, this scheme performs oral imaging of the oral cavity with a positioning device and a positioning element and generates oral imaging results; further, a three-dimensional space coordinate set including the coordinates of the metal reflective device and the positioning element is generated in the three-dimensional space, and a local coordinate set of the optical positioning system including the coordinates of the metal reflective ball and the positioning element is generated in the optical positioning system; a transformation matrix is ​​generated according to the three-dimensional space coordinate set and the learned local coordinate set of the positioning system; a three-dimensional image of the optical positioning device is generated according to the transformation matrix and the optical positioning coordinates of the optical positioning device in the local coordinate system of the optical positioning system, and displayed in the navigation system for real-time navigation; using this method for positioning and navigation, the accuracy is high and the positioning speed is fast.

[0035] Please see Figure 1 , Figure 1 A flowchart of a first oral image positioning and navigation method provided in an embodiment of the present application; the oral image positioning and navigation method comprises:

[0036] Step S100: performing oral imaging on the oral cavity with the positioning device and the positioning element and generating an oral imaging result.

[0037] In the above step S100, oral imaging is performed on the oral cavity with the positioning device and the positioning element and an oral imaging result is generated. The positioning device includes a metal reflective device. For example, the metal reflective device on the positioning device can be a metal reflective ball. The number of metal reflective balls is generally 5 to 11, and the metal reflective balls are evenly distributed on the positioning device, and may not be in the same plane and may have different depths. The positioning element can be a metal reflective patch. It can be explained that the material of the metal reflective device and the positioning element can generally be metal materials such as titanium, iron, and steel.

[0038] Step S101: Calculate the coordinates of the metal reflective device and the positioning element in the three-dimensional image space according to the oral imaging result, and form a three-dimensional space coordinate set.

[0039] In the above step S101, the coordinates of the metal reflective device and the positioning element in the three-dimensional image space are calculated according to the oral imaging result obtained in step S100, and a three-dimensional space coordinate set is formed. For example, in one embodiment, the metal reflective device is 7 metal reflective balls, and the positioning element is a metal reflective patch. According to the coordinates of the 7 metal reflective balls and the metal reflective patch in the three-dimensional space in the oral imaging result, these coordinates can form a three-dimensional space coordinate set, which is usually called ImagePointSet.

[0040] Step S102: Calculate the coordinates of the metal reflective ball and the positioning element in the local coordinate system of the optical positioning system, and form a local coordinate set of the optical positioning system.

[0041] In the above step S102, the coordinates of the metal reflective balls and the positioning element in the local coordinate system of the optical positioning system are calculated to form a local coordinate set of the optical positioning system. Exemplarily, in one embodiment, the metal reflective device is 7 metal reflective balls, and the positioning element is a metal reflective patch. In step S102, the coordinates of the 7 metal reflective balls and the metal reflective patch in the local coordinate system of the optical positioning system are calculated respectively. These coordinates form a local coordinate set of the optical positioning system, which is usually called NDIPointSet.

[0042] Step S103: Generate a transformation matrix according to the three-dimensional space coordinate set and the local coordinate set of the optical positioning system.

[0043] In the above step S103, the three-dimensional space coordinate set obtained in step S101 and step S102 is calculated with the local coordinate set of the optical positioning system to obtain a transformation matrix that converts the local coordinate system of the optical positioning system into the three-dimensional space coordinate system. Exemplarily, the metal reflective device is 7 metal reflective balls, and the positioning element is a metal reflective patch. The three-dimensional space coordinate set of the 7 metal reflective balls and the metal reflective patch and the local coordinate set of the optical positioning system, namely ImagePointSet and NDIPointSet, are calculated respectively, and the ImagePointSet and NDIPointSet are calculated to obtain a transformation matrix, which is usually called MatrixTransform. It can be explained that the calculation algorithm is a registration algorithm, which can be an ICP (Iterative Closest Point) algorithm, that is, an iterative closest point algorithm, or other algorithms well known in the art.

[0044] Step S104: Obtain the optical positioning coordinates of the optical positioning device in the local coordinate system of the optical positioning system.

[0045] In the above step S104, the optical positioning coordinates of the optical positioning device in the local coordinate system of the optical positioning system are obtained, and the optical positioning device is used to capture its position in the optical positioning system and guide the direction in the positioning navigation system. Exemplarily, the optical positioning device can be a probe, and the coordinates of the probe in the optical positioning system are captured. Further, the coordinates are converted into optical positioning coordinates in the local coordinate system of the optical positioning system to obtain the optical positioning coordinates of the probe in the local coordinate system of the optical positioning system.

[0046] Step S105: Calculate the coordinates of the optical positioning device in the three-dimensional image space according to the optical positioning coordinates and the transformation matrix.

[0047] In the above step S105, the transformation matrix obtained in step S103 is multiplied by the optical positioning coordinates in step S104 to obtain the coordinates of the optical positioning device in the three-dimensional space, thereby forming a three-dimensional space image. Exemplarily, the optical positioning coordinates of the probe are multiplied by MatrixTransform to obtain the coordinates of the probe in the three-dimensional space, thereby forming a three-dimensional space image and guiding the direction in the positioning and navigation system.

[0048] It can be seen that the flow chart of the first oral image positioning and navigation method provided in the embodiment of the present application performs oral imaging of an oral cavity with a positioning device and a positioning element and generates an oral imaging result; further, generates a three-dimensional space coordinate set including the coordinates of the metal reflective device and the positioning element in the three-dimensional space, and generates an optical positioning system local coordinate set including the coordinates of the metal reflective ball and the positioning element in the optical positioning system; generates a transformation matrix based on the three-dimensional space coordinate set and the learned positioning system local coordinate set; generates a three-dimensional image of the optical positioning device based on the transformation matrix and the optical positioning coordinates of the optical positioning device in the local coordinate system of the optical positioning system, and displays it in the navigation system for real-time navigation; using this method for positioning and navigation has high accuracy and fast positioning speed, so that oral surgery can be performed better.

[0049] Please see Figure 2 , Figure 2 This is a position relationship diagram in the application scenario of the oral image positioning and navigation method provided in the embodiment of the present application. Figure 2 As shown in , the positioning element 203 is installed in the adjacent area 201 of the surgical execution area, and the positioning device 200 is installed on the opposite side of the positioning element 203; the positioning device 200 also includes a positioning guide plate 220 and an infrared reflective device 210 for determining the coordinate system of the optical positioning system; the infrared reflective device 210 is installed on the positioning guide plate. Exemplarily, the position relationship diagram of the oral image positioning and navigation method includes: the positioning device 200, the adjacent area 201 of the surgical execution area, and the surgical execution area 202, wherein the positioning device 200 includes an infrared reflective ball 210, a positioning guide plate 220 and a metal reflective ball 230. The surgical area 202 is on the left side of the mandible, and the metal reflective ball 230 is installed on the right side of the mandible; the positioning element 203 is installed in the adjacent area 201 of the surgical area, which is the crown of the adjacent tooth in this example; the positioning device 200 is installed on the opposite side of the positioning element; the positioning device also includes a positioning guide plate 220 and an infrared reflective ball 210 for determining the coordinate system of the optical positioning system. The positioning guide plate 220 can usually cover one-third of the mandible or maxilla, and the infrared reflective ball 210 is installed on the positioning guide plate 220. Through the installation of the positioning device and the positioning element, the imaging positioning accuracy of the surgical area is improved, which is conducive to forming a high-precision three-dimensional navigation image.

[0050] Please see Figure 3 , Figure 3 A flowchart of a second oral image positioning and navigation method provided in an embodiment of the present application, the oral image positioning and navigation method comprising:

[0051] Step S300: obtaining the offset of the metal reflective device and the positioning element from the coordinate origin in the three-dimensional space from the imaging result.

[0052] In the above step S300, the offset of the metal reflective device and the positioning element from the coordinate origin in the three-dimensional space is obtained from the imaging result. Exemplarily, the metal reflective device is 7 metal reflective balls, the positioning element is a metal reflective patch, and the oral imaging result can be CBCT. The offset of the CBCT image relative to the origin of the three-dimensional image space is directly obtained. In this embodiment, the offset is (100, 200, 300).

[0053] Step S301: converting the oral cavity imaging result into a grayscale image.

[0054] In the above step S301, the oral imaging result is converted into a grayscale image. Exemplarily, the metal reflective device is 7 metal reflective balls, and the positioning element is a metal reflective patch; the oral imaging result can be a CBCT image; the grayscale image can be a commonly used image format such as JPEG image format or PNG image format; the DICOM format of the CBCT image is converted into a commonly used image format such as JPEG or PNG.

[0055] Step S302: Calculate the center coordinates of the metal reflective device and the positioning element on the grayscale image.

[0056] In the above step S302, the center coordinates of the metal reflective device and the positioning element on the grayscale image are calculated. Exemplarily, the metal reflective device is 7 metal reflective balls, and the positioning element is a metal reflective patch; the oral imaging result is a CBCT image; in this embodiment, the pixel interval in the CBCT image is 0.25 mm, and the intervals in the three directions of X, Y and Z are all 0.25 mm. The size of the three-dimensional image is 640×640×400, that is, the pixel size of each picture is 640×640, and there are 400 pictures in total. The format of the CBCT image sequence is converted from the DICOM format to the JPEG image format. The diameter of the metal ball in this embodiment is 3.5 mm, and the interval between pixels is 0.25 mm. According to the diameter of the metal ball divided by the interval between pixels, that is, 3.5÷0.25=14, it is concluded that the diameter of the metal reflective ball is 14 pixels in length; further, the Hough circle transform in OpenCV is used to identify the metal reflective ball and return the coordinates of the center of the circle, and the radius of the circle identified by the Hough circle finding function is set to the radius of the metal reflective ball, that is, 7 pixels in length.

[0057] Step S303: Calculate the coordinates of the metal reflective device and the positioning element in the three-dimensional image space according to the image sequence number, offset and center coordinates of the imaging result.

[0058] In the above step S303, the coordinates of the metal reflective device and the positioning element in the three-dimensional image space are calculated according to the image serial number, offset and center coordinates of the imaging result. For example, the metal reflective device is 7 metal reflective balls, and the positioning element is a metal reflective patch; the oral imaging result is a CBCT image, the pixel interval is 0.25mm, the pixel size of each image is 640×640, there are 400 images in total, each image has a fixed serial number, and the serial number can represent the oral fixed position, such as 00 1 represents the part close to the mandible, and 400 represents the part close to the maxilla; the Hough circle transform in OpenCV is used to identify the metal reflective ball and return the coordinates of the center of the circle, that is, the center coordinates; it is found that there is a circle with a diameter of 14 pixels in the JPEG image with serial number 320, and its center coordinates are (500, 450), and the offset of the CBCT image relative to the origin of the three-dimensional coordinates is (100, 200, 300), then the coordinates of the metal reflective ball and the positioning patch in the three-dimensional space are calculated according to the serial number, center coordinates, and offset.

[0059] It can be seen that the flow chart of the second oral image positioning and navigation method provided in the embodiment of the present application calculates the coordinates of the metal reflective device and the positioning element in three-dimensional space based on the oral imaging results. The coordinates in the three-dimensional space calculated in this way are accurate, do not require manual acquisition of coordinates, and are convenient and fast.

[0060] Please see Figure 4 , Figure 4 A flowchart of a third oral image positioning and navigation method provided in an embodiment of the present application, the oral image positioning and navigation method comprising:

[0061] Step S400: determining the horizontal coordinate in the three-dimensional image space as the sum of the product of the horizontal coordinate of the center coordinate and the pixel interval of the imaging result and the horizontal coordinate of the offset.

[0062] Step S401: Determine the ordinate in the three-dimensional image space as the sum of the product of the ordinate of the center coordinate and the pixel interval of the imaging result and the ordinate of the offset.

[0063] Step S402: Determine the vertical coordinate in the three-dimensional image space as the sum of the product of the picture sequence number of the imaging result and the pixel interval of the imaging result and the vertical coordinate of the offset.

[0064] In the above steps S400, S401 and S402, the horizontal coordinate in the three-dimensional image space is determined according to the product of the horizontal coordinate of the center coordinate and the pixel interval of the imaging result and the sum of the horizontal coordinate of the offset; the vertical coordinate in the three-dimensional image space is determined according to the product of the vertical coordinate of the center coordinate and the pixel interval of the imaging result and the sum of the vertical coordinate of the offset; and the vertical coordinate in the three-dimensional image space is determined according to the product of the picture sequence number of the imaging result and the pixel interval of the imaging result and the sum of the vertical coordinate of the offset. Exemplarily, the metal reflective device is 7 metal reflective balls, and the positioning element is a metal reflective patch; the oral imaging result is a CBCT image with a pixel interval of 0.25 mm, a pixel size of each picture of 640×640, a total of 400 pictures, each picture has a fixed serial number, and the serial number can represent the fixed position of the oral cavity, such as 001 represents close to the mandibular part, and 400 represents close to the maxillary part; the Hough circle transform in OpenCV is used to identify the metal reflective balls and return the coordinates of the center of the circle, that is, the center coordinates; it is found that there is a circle with a diameter of 14 pixels in the JPEG image with serial number 320, and its center coordinates are (500, 450), and the offset of the CBCT image relative to the three-dimensional coordinate origin is (100, 200, 300), then the coordinate values ​​of the metal ball in the three-dimensional image space are (100+500×0.25, 200+450×0.25, 300+320×0.25).

[0065] Please see Figure 5 , Figure 5 A flowchart of a fourth oral image positioning and navigation method provided in an embodiment of the present application, the oral image positioning and navigation method comprising:

[0066] Step S500: establishing a local coordinate system of the optical positioning system of the positioning device according to the positioning guide.

[0067] In the above step S500: a local coordinate system of the optical positioning system of the positioning device is established according to the physical structure of the positioning guide. For example, when the local coordinate system of the positioning device is set in the optical positioning system, the origin, horizontal axis, longitudinal axis and vertical axis of the coordinate system are set to be consistent with the orientation of the physical structure of the positioning guide.

[0068] Step S501: obtaining the coordinates of the metal reflective device in the local coordinate system of the optical positioning system according to the positioning device.

[0069] In the above step S501, the coordinates of the metal reflective device and the positioning element are obtained in the local coordinate system of the optical positioning system established in step S500. Exemplarily, the metal reflective device is 7 metal reflective balls; the difference between the first metal reflective ball and the origin in the positive direction of the horizontal axis is 20, the difference in the positive direction of the vertical axis is 50, and the difference in the negative direction of the longitudinal axis is 80. Then, the value of the metal reflective ball in the local coordinate system of the optical positioning system of the positioning device is (20, 50, -80), and the remaining 6 metal reflective balls are calculated in the same principle.

[0070] Step S502: Obtaining the coordinates of the positioning element in the local coordinate system of the optical positioning system according to the coordinates of the positioning element in the optical positioning system and the coordinates of the positioning guide in the optical positioning system.

[0071] In the above step S502, the coordinates of the positioning element in the local coordinate system of the optical positioning system are determined by the coordinates of the positioning element in the optical positioning system and the coordinates of the positioning guide in the optical positioning system. Exemplarily, a tool with an optical positioning device is used to capture the coordinates of the positioning element in the optical positioning system. The tool may be a probe, and the positioning element may be a metal reflective patch. That is, a probe with an optical positioning device is used to capture the coordinate value of the center of the metal reflective patch in the optical positioning system. If the position in the optical positioning system captured by the probe is the probe matrix MatrixProbe, the position of the positioning guide in the optical positioning system is the guide matrix MatrixGuide, and the inverse matrix of the guide matrix is ​​InverseMatrixGuide, then the coordinate matrix of the probe in the local coordinate system of the optical positioning system is the inverse matrix of the guide matrix multiplied by the probe matrix, that is, the coordinate matrix of the probe in the local coordinate system of the optical positioning system P = InverseMatrixGuide × MatrixProbe, and then these centers are added to the NDIPointSet point set.

[0072] It can be seen that the flow chart of the fourth oral image positioning and navigation method provided in the embodiment of the present application respectively obtains the coordinates of the metal reflective device and the positioning element in the local coordinate system of the optical positioning system. The intermediate calculation process is matrix operation. The captured coordinates are consistent with the actual oral situation and have high accuracy.

[0073] Please see Figure 6 , Figure 6A schematic diagram of the local coordinate system of the optical positioning system of the oral image positioning and navigation method provided in the embodiment of the present application, wherein the method for establishing the local coordinate system of the optical positioning system includes: identifying the center of the infrared reflective device in the optical positioning system as the coordinate origin of the local coordinate system of the optical positioning system of the positioning device; the horizontal axis, vertical axis and vertical axis of the positioning device in the optical positioning system are consistent with the preset orientation of the positioning device. Exemplarily, the coordinate system includes an origin O, a horizontal axis X, a vertical axis Y and a vertical axis Z, and the infrared reflective device on the positioning guide plate is 4 infrared reflective balls, and the centers of the four infrared reflective balls are identified as the coordinate origin in the optical positioning system; the horizontal axis, vertical axis and vertical axis of the positioning device in the optical positioning system are consistent with the preset orientation of the physical structure of the positioning device. The local coordinate system of the optical positioning system is established by this coordinate system setting method, and the coordinates in all optical positioning systems are unified into the local coordinate system of the optical positioning system, which is convenient for later calculations and can establish a coordinate system that is consistent with the actual oral situation.

[0074] In an optional embodiment, after calculating the coordinates of the optical positioning device in the three-dimensional image space, the three-dimensional space image of the optical positioning device is transmitted and displayed on the navigation system; the three-dimensional image guides the oral surgery equipment robot arm to perform the surgery.

[0075] Please see Figure 7 , Figure 7 Schematic diagram of the oral image positioning and navigation system module provided in the embodiment of the present application. The oral image positioning and navigation system in this embodiment is used to perform each step in the above method embodiment, and the oral image positioning and navigation system 700 includes: an oral imaging module 701, a three-dimensional space coordinate set generation module 702, an optical positioning system local coordinate set generation module 703, a conversion matrix generation module 704, and an optical positioning device navigation module 705.

[0076] The oral cavity imaging module 701 is used to perform oral cavity imaging on an oral cavity with a positioning device and a positioning element and generate an oral cavity imaging result; wherein the positioning device includes a metal reflective device.

[0077] The three-dimensional space coordinate set generation module 702 is used to calculate the coordinates of the metal reflective device and the positioning element in the three-dimensional image space according to the oral imaging result output by the oral imaging module 701, and form a three-dimensional space coordinate set.

[0078] The optical positioning system local coordinate set generation module 703 is used to calculate the coordinates of the metal reflective ball and the positioning element in the optical positioning system local coordinate system and form the optical positioning system local coordinate set.

[0079] The transformation matrix generation module 704 is used to generate a transformation matrix based on the three-dimensional space coordinate set generated by the three-dimensional space coordinate set generation module 702 and the optical positioning system local coordinate set generated by the optical positioning system local coordinate set generation module 703; wherein the transformation matrix is ​​used to convert the optical positioning system local coordinate system into the three-dimensional space coordinate system.

[0080] The optical positioning device navigation module 705 is used to obtain the optical positioning coordinates of the optical positioning device in the local coordinate system of the optical positioning system; calculate the coordinates of the optical positioning device in the three-dimensional image space according to the optical positioning coordinates and the transformation matrix generated by the transformation matrix generation module 704; wherein the optical positioning device is used to capture its position in the optical positioning system and guide the direction in the positioning and navigation system.

[0081] In an optional embodiment, the calculation of the coordinates of the metal reflective device and the positioning element in the three-dimensional image space based on the oral imaging results output by the oral imaging module 701 includes: obtaining the offset of the metal reflective device and the positioning element from the coordinate origin in the three-dimensional space from the imaging results output by the oral imaging module 701; the coordinate origin is determined by the infrared reflective device; converting the oral imaging results output by the oral imaging module 701 into a grayscale image; calculating the center coordinates of the metal reflective device and the positioning element on the grayscale image; and calculating the coordinates of the metal reflective device and the positioning element in the three-dimensional image space based on the image serial number of the imaging results output by the oral imaging module 701, the offset and the center coordinates.

[0082] In an optional embodiment, the three-dimensional space coordinate set generation module 702 calculates the coordinates of the metal reflective device and the positioning element in the three-dimensional image space according to the picture serial number of the imaging result output by the oral imaging module 701, the offset and the center coordinate, including: determining the horizontal coordinate in the three-dimensional image space as the product of the horizontal coordinate of the center coordinate and the pixel interval of the imaging result output by the oral imaging module 701 and the sum of the horizontal coordinate of the offset; determining the vertical coordinate in the three-dimensional image space as the product of the vertical coordinate of the center coordinate and the pixel interval of the imaging result output by the oral imaging module 701 and the sum of the vertical coordinate of the offset; determining the vertical coordinate in the three-dimensional image space as the product of the picture serial number of the imaging result and the pixel interval of the imaging result output by the oral imaging module 701 and the sum of the vertical coordinate of the offset.

[0083] In an optional embodiment, the optical positioning system local coordinate set generation module 703 calculates the coordinates of the metal reflective ball and the positioning element in the local coordinate system of the optical positioning system, including: establishing the local coordinate system of the optical positioning system of the positioning device according to the positioning guide; obtaining the coordinates of the metal reflective device in the local coordinate system of the optical positioning system according to the positioning device; obtaining the coordinates of the positioning element in the local coordinate system of the optical positioning system according to the coordinates of the positioning element in the optical positioning system and the coordinates of the positioning guide in the optical positioning system.

[0084] In an optional embodiment, the optical positioning system local coordinate set generation module 703 establishes the optical positioning system local coordinate system of the positioning device according to the positioning guide plate, including: identifying the center of the infrared reflection device in the optical positioning system as the coordinate origin of the optical positioning system local coordinate system of the positioning device; the horizontal axis, longitudinal axis and vertical axis of the positioning device in the optical positioning system are consistent with the preset orientation of the positioning device.

[0085] In an optional embodiment, after the optical positioning device navigation module 705 calculates the coordinates of the optical positioning device in the three-dimensional image space, the method further includes: the optical positioning device navigation module 705 transmits the three-dimensional space image of the optical positioning device and displays it on the navigation system; the three-dimensional image guides the oral surgery equipment robot arm to perform surgery.

[0086] See also Figure 8 The electronic device 800 provided by the embodiment of the present application includes: a processor 801 and a memory 802, wherein the memory 802 stores machine-readable instructions executable by the processor 801, and when the machine-readable instructions are executed by the processor 801, the above method is executed.

[0087] Based on the same inventive concept, an embodiment of the present application further provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer program instructions, and when the computer program instructions are read and executed by a processor, the steps in any of the above implementation methods are executed.

[0088] The computer-readable storage medium may be a random access memory (RAM), a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or any other medium that can store program codes. The storage medium is used to store the program, and the processor executes the program after receiving the execution instruction. The method executed by the electronic terminal of the process definition disclosed in any embodiment of the present invention can be applied to the processor or implemented by the processor.

[0089] In the embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are merely schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some communication interfaces, and the indirect coupling or communication connection of the devices or units can be electrical, mechanical or other forms.

[0090] In addition, the units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0091] Furthermore, the functional modules in the various embodiments of the present application may be integrated together to form an independent part, or each module may exist separately, or two or more modules may be integrated to form an independent part.

[0092] Alternatively, the present invention may be implemented in whole or in part by software, hardware, firmware, or any combination thereof. When implemented by software, the present invention may be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present invention is generated in whole or in part.

[0093] The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions may be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means.

[0094] In this article, relational terms such as first and second, etc. are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the statement "include..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.

[0095] The above description is only an embodiment of the present application and is not intended to limit the protection scope of the present application. For those skilled in the art, the present application may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. An oral image positioning and navigation method, characterized in that: The method is applied to a positioning navigation system of an oral surgery device to determine a surgery execution site; the method comprises: Performing oral imaging on an oral cavity with a positioning device and a positioning element and generating an oral imaging result; wherein the positioning device comprises a metal reflective device, a positioning guide plate and an infrared reflective device for determining a coordinate system of an optical positioning system; the infrared reflective device is mounted on the positioning guide plate; According to the oral imaging result, the coordinates of the metal reflective device and the positioning element in the three-dimensional image space are calculated to form a three-dimensional space coordinate set; The center of the infrared reflective device is identified in the optical positioning system as the coordinate origin of the local coordinate system of the optical positioning system of the positioning device; the horizontal axis, the longitudinal axis and the vertical axis of the positioning device in the optical positioning system are consistent with the preset orientation of the positioning device; the coordinates of the metal reflective device in the local coordinate system of the optical positioning system are obtained according to the positioning device; the coordinates of the positioning element in the local coordinate system of the optical positioning system are obtained according to the coordinates of the positioning element in the optical positioning system and the coordinates of the positioning guide in the optical positioning system; Generate a transformation matrix according to the three-dimensional space coordinate set and the local coordinate set of the optical positioning system; wherein the transformation matrix is ​​used to transform the local coordinate system of the optical positioning system into the three-dimensional space coordinate system; and Acquire the optical positioning coordinates of the optical positioning device in the local coordinate system of the optical positioning system; calculate the coordinates of the optical positioning device in the three-dimensional image space according to the optical positioning coordinates and the transformation matrix; Wherein, the optical positioning device is used to capture its position in the optical positioning system and guide the direction in the positioning and navigation system; Wherein, the optical positioning device includes a probe, and the optical positioning element includes a metal reflective patch; The acquiring of the optical positioning coordinates of the optical positioning device in the local coordinate system of the optical positioning system comprises: capturing the coordinates of the probe in the optical positioning system, and converting the coordinates of the probe in the optical positioning system into the coordinates in the local coordinate system of the optical positioning system; The method of calculating the coordinates of the optical positioning device in the three-dimensional image space based on the optical positioning coordinates and the transformation matrix includes: using the probe to capture the center of the metal reflective patch in the probe matrix MatrixProbe of the optical positioning system, and the guide plate matrix MatrixGuide of the positioning guide in the optical positioning system; multiplying the inverse matrix of the guide plate matrix with the probe matrix to obtain the coordinate matrix of the probe in the local coordinate system of the optical positioning system.

2. The method according to claim 1, characterized in that in, The positioning element is installed in an area adjacent to a surgical area, and the positioning device is installed on an opposite side of the positioning element.

3. The method according to claim 2, characterized in that The step of calculating the coordinates of the metal reflective device and the positioning element in the three-dimensional image space according to the oral imaging result comprises: Obtaining the offset of the metal reflective device and the positioning element from the coordinate origin in three-dimensional space from the imaging result; the coordinate origin is determined by the position of the center of the infrared reflective device in the optical positioning system; Converting the oral cavity imaging result into a grayscale image; Calculating the center coordinates of the metal reflective device and the positioning element on the grayscale image; and The coordinates of the metal reflective device and the positioning element in the three-dimensional image space are calculated according to the image sequence number of the imaging result, the offset and the center coordinates.

4. The method according to claim 3, characterized in that The calculating of the coordinates of the metal reflective device and the positioning element in the three-dimensional image space according to the image sequence number of the imaging result, the offset and the center coordinates includes: Determine the abscissa in the three-dimensional image space as the sum of the product of the abscissa of the center coordinate and the pixel interval of the imaging result and the abscissa of the offset; Determine the ordinate in the three-dimensional image space as the sum of the product of the ordinate of the center coordinate and the pixel interval of the imaging result and the ordinate of the offset; The vertical coordinate in the three-dimensional image space is determined to be the sum of the product of the picture sequence number of the imaging result and the pixel interval of the imaging result and the vertical coordinate of the offset.

5. The method according to claim 1, characterized in that: After calculating the coordinates of the optical positioning device in the three-dimensional image space, the method further includes: transmitting and displaying the three-dimensional space image of the optical positioning device on the navigation system; the three-dimensional image guides the oral surgery equipment robot arm to perform surgery.

6. A positioning navigation system, characterized in that: The positioning and navigation system comprises: An oral imaging module, used for performing oral imaging of an oral cavity with a positioning device and a positioning element and generating an oral imaging result; wherein the positioning device comprises a metal reflective device, a positioning guide plate and an infrared reflective device for determining a coordinate system of an optical positioning system; the infrared reflective device is mounted on the positioning guide plate; A three-dimensional space coordinate set generation module, used to calculate the coordinates of the metal reflective device and the positioning element in the three-dimensional image space according to the oral imaging result, and form a three-dimensional space coordinate set; An optical positioning system local coordinate set generation module is used to identify the center of the infrared reflective device in the optical positioning system as the coordinate origin of the local coordinate system of the optical positioning system of the positioning device; the horizontal axis, the longitudinal axis and the vertical axis of the positioning device in the optical positioning system are consistent with the preset orientation of the positioning device; the coordinates of the metal reflective device in the local coordinate system of the optical positioning system are obtained according to the positioning device; the coordinates of the positioning element in the local coordinate system of the optical positioning system are obtained according to the coordinates of the positioning element in the optical positioning system and the coordinates of the positioning guide in the optical positioning system; A transformation matrix generation module, for generating a transformation matrix according to the three-dimensional space coordinate set and the local coordinate set of the optical positioning system; wherein the transformation matrix is ​​used to transform the local coordinate system of the optical positioning system into the three-dimensional space coordinate system; wherein the optical positioning device includes a probe, and the optical positioning element includes a metal reflective patch; obtaining the optical positioning coordinates of the optical positioning device in the local coordinate system of the optical positioning system, comprising: capturing the coordinates of the probe in the optical positioning system, and transforming the coordinates of the probe in the optical positioning system into the coordinates in the local coordinate system of the optical positioning system; calculating the coordinates of the optical positioning device in the three-dimensional image space according to the optical positioning coordinates and the transformation matrix, comprising: using the probe to capture the center of the metal reflective patch in the probe matrix MatrixProbe of the optical positioning system, and the guide plate matrix MatrixGuide of the positioning guide in the optical positioning system; multiplying the inverse matrix of the guide plate matrix with the probe matrix to obtain the coordinate matrix of the probe in the local coordinate system of the optical positioning system; and An optical positioning device navigation module is used to obtain the optical positioning coordinates of the optical positioning device in the local coordinate system of the optical positioning system; calculate the coordinates of the optical positioning device in the three-dimensional image space according to the optical positioning coordinates and the transformation matrix; wherein the optical positioning device is used to capture its position in the optical positioning system and guide the direction in the positioning navigation system.

7. An electronic device, characterized in that: The electronic device comprises a memory and a processor, wherein the memory stores program instructions, and when the processor runs the program instructions, the steps in the method according to any one of claims 1 to 5 are executed.

8. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer program instructions, and when the computer program instructions are executed by a processor, the steps in the method according to any one of claims 1 to 5 are executed.

Images