Navigation registration method, device, equipment and storage medium

Abstract

The invention discloses a navigation registration method and device, equipment and a storage medium. The navigation registration method comprises the steps that an oral cavity scanning image of a target object and point cloud data of a target maxillofacial surface are acquired, and the point cloud data of the target maxillofacial surface comprise tooth point cloud data and current point cloud data of auxiliary components; extracting a target maxillofacial scanning image corresponding to the target maxillofacial surface from the oral cavity scanning image; performing registration based on the target maxillofacial scanning image and the point cloud data of the target maxillofacial to obtain a first conversion matrix; extracting current point cloud data of the auxiliary part from the point cloud data of the target maxillofacial surface; performing registration based on the current point cloud data of the auxiliary component and the pre-configured point cloud data of the auxiliary component to obtain a second conversion matrix; and determining a third conversion matrix based on the first conversion matrix and the second conversion matrix, wherein the third conversion matrix is used for indicating a conversion relationship between the oral cavity scanning image of the target object and the navigation component. And the navigation registration efficiency and accuracy are improved.

Description

Technical Field

[0001] The present invention relates to the field of oral technology, and in particular to a navigation registration method, device, equipment and storage medium. Background Art

[0002] In the field of oral technology, surgical robots are increasingly used and play an important role in dental implant planning and dental restoration. In the process of dental implantation based on oral cone beam computed tomography (CBCT) images, the surgical robot is connected to the splint attachment through the navigation component, and the splint attachment is connected to the tooth to achieve navigation positioning of the tooth.

[0003] Although surgical robots can be used to achieve navigation and positioning of teeth, there are still problems such as inaccurate navigation and positioning of teeth by surgical robots and low alignment efficiency. Summary of the invention

[0004] The present invention provides a navigation registration method, device, equipment and storage medium to solve the problems of inaccurate navigation registration and low registration efficiency of a surgical robot.

[0005] According to one aspect of the present invention, a navigation registration method is provided, the method comprising:

[0006] Acquire an oral scan image of a target object and point cloud data of a target maxillofacial region, wherein the point cloud data of the target maxillofacial region includes tooth point cloud data and current point cloud data of an auxiliary component, and the auxiliary component is used to connect to a navigation component;

[0007] Extracting a target maxillofacial scan image corresponding to the target maxillofacial region from the oral scan image;

[0008] Performing registration based on the target maxillofacial scan image and the target maxillofacial point cloud data to obtain a first transformation matrix;

[0009] Extracting current point cloud data of the auxiliary component from the point cloud data of the target maxillofacial region; performing registration based on the current point cloud data of the auxiliary component and the pre-configured point cloud data of the auxiliary component to obtain a second transformation matrix;

[0010] A third conversion matrix is ​​determined based on the first conversion matrix and the second conversion matrix, and the third conversion matrix is ​​used to indicate a conversion relationship between the oral scan image of the target object and the navigation component.

[0011] According to another aspect of the present invention, a navigation registration device is provided, the device comprising:

[0012] The oral scan image and target maxillofacial point cloud data acquisition module is used to acquire the oral scan image and target maxillofacial point cloud data of the target object, wherein the target maxillofacial point cloud data includes tooth point cloud data and current point cloud data of auxiliary components, and the auxiliary components are used to connect the navigation components;

[0013] A target maxillofacial scan image extraction module is used to extract a target maxillofacial scan image corresponding to the target maxillofacial face from the oral scan image;

[0014] A first conversion matrix determination module is used to perform registration based on the target maxillofacial scan image and the target maxillofacial point cloud data to obtain a first conversion matrix;

[0015] A second conversion matrix determination module is used to extract the current point cloud data of the auxiliary component from the point cloud data of the target maxillofacial region; perform registration based on the current point cloud data of the auxiliary component and the pre-configured point cloud data of the auxiliary component to obtain a second conversion matrix;

[0016] The third conversion matrix determination module is used to determine a third conversion matrix based on the first conversion matrix and the second conversion matrix, and the third conversion matrix is ​​used to indicate the conversion relationship between the oral scan image of the target object and the navigation component.

[0017] According to another aspect of the present invention, there is provided an electronic device, the electronic device comprising:

[0018] at least one processor; and

[0019] a memory communicatively connected to at least one processor; wherein,

[0020] The memory stores a computer program that can be executed by at least one processor. The computer program is executed by at least one processor so that the at least one processor can execute the navigation and registration method provided by any embodiment of the present invention.

[0021] According to another aspect of the present invention, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the navigation and registration method provided by any embodiment of the present invention when executed.

[0022] The technical solution of the embodiment of the present invention is to obtain the oral scan image and the point cloud data of the target maxillofacial face of the target object, wherein the point cloud data of the target maxillofacial face includes the tooth point cloud data and the current point cloud data of the auxiliary component, and the auxiliary component is used to connect the navigation component; extract the target maxillofacial scan image corresponding to the target maxillofacial face from the oral scan image; perform alignment based on the target maxillofacial scan image and the point cloud data of the target maxillofacial face to obtain a first transformation matrix; extract the current point cloud data of the auxiliary component from the point cloud data of the target maxillofacial face; perform alignment based on the current point cloud data of the auxiliary component and the pre-configured point cloud data of the auxiliary component to obtain a second transformation matrix; determine the third transformation matrix based on the first transformation matrix and the second transformation matrix, and the third transformation matrix is ​​used to indicate the transformation relationship between the oral scan image of the target object and the navigation component, thereby solving the problems of inaccurate navigation registration and low registration efficiency of the surgical robot and improving the efficiency and accuracy of the navigation registration of the surgical robot.

[0023] It should be understood that the contents described in this section are not intended to identify the key or important features of the embodiments of the present invention, nor are they intended to limit the scope of the present invention. Other features of the present invention will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0025] Figure 1 is a flow chart of a navigation registration method provided by Embodiment 1 of the present invention;

[0026] Figure 2 is a structural schematic diagram of a navigation and registration device provided in Embodiment 3 of the present invention;

[0027] Figure 3 It is a structural schematic diagram of an electronic device provided in Embodiment 3 of the present invention. DETAILED DESCRIPTION

[0028] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0029] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0030] Embodiment 1

[0031] Figure 1 This is a flowchart of a navigation registration method provided in the first embodiment of the present invention. This embodiment can be applied to improve the accuracy of navigation registration of surgical robots, and is applied to surgical robots in the oral field. The method can be executed by a navigation registration device, and the navigation registration device can be implemented in the form of hardware and / or software. The navigation registration device can be configured in the electronic device provided in the embodiment of the present invention. The electronic device can be a server, a computer, a mobile terminal or a medical device, etc. For example, the mobile terminal can be a mobile phone, a tablet computer, etc. For example, the medical device can be a device equipped with a processor, such as an oral medical device, and the oral medical device can be equipped with an oral scanning device and / or an oral surgical instrument, etc. Figure 1 As shown, the method includes:

[0032] S110, obtaining an oral scan image of the target object and point cloud data of the target maxillofacial region, wherein the point cloud data of the target maxillofacial region includes tooth point cloud data and current point cloud data of an auxiliary component, and the auxiliary component is used to connect to the navigation component.

[0033] The process of implementing the navigation registration method includes a preoperative planning scene and a surgical scene. The oral scan image of the target object can be acquired in the preoperative planning scene, and the dental implant planning can be performed before the operation through the oral scan image. Through the navigation registration method of the present invention, it is not necessary to acquire the oral scan image of the target object again in the surgical scene, thereby reducing the damage to the human body during the acquisition of the oral scan image. Among them, the oral scan image of the target object does not include auxiliary components.

[0034] In this embodiment, the oral scan image can be understood as an image obtained by scanning the oral cavity through an oral scanning device, for example, it can include an oral CBCT image, which can be used in oral fields such as dental implant planning and dental periodontal disease diagnosis. Optionally, the oral scanning device can be an oral CBCT. Among them, the oral scan image can include an oral scan image of the tooth area and an oral scan image of the background area, wherein the background area can include but is not limited to the gum area and the gum area. The oral scan image can be obtained by scanning the oral cavity through the oral scanning device, and can also be obtained by reading from a database, wherein the database stores oral scan images of multiple target objects, and by inputting user information, the oral scan image is matched with the target object information in the database, and when the match is successful, the matched oral scan image is read. The target maxillofacial surface can be understood as the maxillofacial surface operated by the surgical robot, which can be the maxillofacial surface or the mandibular surface. The auxiliary component can be understood as a component for connecting the navigation component, and the auxiliary component can be fixed on the target maxillofacial surface. The auxiliary component can include a connecting splint, and the connecting splint includes holes for connecting the navigation component. The navigation component can be understood as a component that can provide navigation information for the surgical robot, and the surgical robot can achieve accurate operation of dental implantation on the target maxillofacial surface through the navigation component. The current point cloud data of the auxiliary component can be understood as the point cloud data of the auxiliary component in the target jaw surface. After the target object's teeth wear the auxiliary component, the point cloud data of the target jaw surface is collected, and the point cloud data of the target jaw surface includes the tooth point cloud data and the current point cloud data of the auxiliary component.

[0035] S120, extracting a target maxillofacial scan image corresponding to the target maxillofacial region from the oral scan image.

[0036] In the present embodiment, the target maxillofacial scan image can be understood as a local oral scan image corresponding to the target maxillofacial. When the target maxillofacial is the maxillary face, the target maxillofacial scan image can be an oral scan image corresponding to the maxillofacial. When the target maxillofacial is the mandibular face, the target maxillofacial scan image can be an oral scan image corresponding to the mandibular face. When the target maxillofacial is different, the target maxillofacial scan image corresponding to the target maxillofacial is also different. In order to obtain the target maxillofacial scan image corresponding to the target maxillofacial, it can be extracted from the oral scan image. The target maxillofacial scan image corresponding to the target maxillofacial can be manually divided by the operator, and the operator can extract the target maxillofacial scan image corresponding to the target maxillofacial from the oral scan image according to the actual situation of the target object. The target maxillofacial scan image corresponding to the target maxillofacial can also be obtained through a target maxillofacial scan image recognition model, for example, the oral scan image is input into a pre-trained target maxillofacial scan image recognition model for recognition, and the target maxillofacial scan image corresponding to the target maxillofacial is obtained. The target maxillofacial scan image extraction model includes but is not limited to a neural network model.

[0037] Optionally, determine the maxillofacial type of the target maxillofacial face; divide the oral scan image into a maxillofacial scan image and a mandibular scan image; and determine the maxillofacial scan image or the mandibular scan image as the target maxillofacial scan image based on the maxillofacial type.

[0038] Specifically, the maxillofacial type may include the maxillary face and the mandibular face. The maxillofacial scan image may be understood as the oral scan image corresponding to the maxillofacial face, and the mandibular scan image may be understood as the oral scan image corresponding to the mandibular face. The maxillofacial type of the target maxillofacial face may be manually divided by an operator, and the operator may select the maxillofacial type of the target maxillofacial face according to the actual situation of the target object. Optionally, before collecting the point cloud data of the target maxillofacial face, an operation page of the oral scan image is displayed, and the maxillofacial type of the target maxillofacial face is configured in response to the setting operation of the operator on the operation page. Accordingly, the collected point cloud data carries the maxillofacial type. For example, taking the maxillofacial face as the target maxillofacial face as an example, the operator may select the maxillofacial type of the target maxillofacial face as the maxillofacial face according to the actual situation of the target object, input the oral scan image of the target object into a pre-trained target maxillofacial scan image recognition model for identification, obtain the maxillofacial scan image, and determine the maxillofacial scan image as the target maxillofacial scan image.

[0039] It is understandable that if the operator does not pre-set the maxillofacial type of the target maxillofacial face, the maxillofacial type corresponding to the point cloud data cannot be directly known. In this embodiment, the maxillofacial type of the target maxillofacial face is determined by automatically identifying the point cloud data of the target maxillofacial face. Optionally, the maxillofacial type of the target maxillofacial face can also be obtained through a maxillofacial type recognition model. For example, the point cloud data of the target maxillofacial face can be input into a pre-trained maxillofacial type recognition model for recognition to obtain the maxillofacial type of the target maxillofacial face. The maxillofacial type recognition model includes but is not limited to a neural network model. To determine the maxillofacial type of the target maxillofacial face, the oral scan image is divided into an upper maxillofacial scan image and a lower maxillofacial scan image. The target maxillofacial scan image can be determined according to the maxillofacial type of the target maxillofacial face.

[0040] In some embodiments, the operator can determine the target maxillofacial region according to the actual situation of the target object. Before obtaining the oral scanning image of the target object, the operator can set the device parameters of the oral scanning device, and the oral scanning device scans the target maxillofacial region of the target object to obtain the target maxillofacial scanning image.

[0041] Optionally, determining the maxillofacial type of the target maxillofacial face includes: determining the density distribution of the point cloud data of the target maxillofacial face, matching the density distribution of the point cloud data of the target maxillofacial face with the preset maxillary face density distribution and the preset mandibular face density distribution, respectively, to obtain a matching result; determining the maxillofacial type of the target maxillofacial face based on the matching result, the maxillofacial type including the maxillary face and the mandibular face.

[0042] Specifically, the matching result can be understood as the result obtained by matching the density distribution of the point cloud data of the target maxillofacial face with the preset maxillofacial face density distribution and the preset mandibular face density distribution. Since the density distribution of the point cloud data of the maxillofacial face is different from that of the point cloud data of the mandibular face, the density distribution of the point cloud data of the target maxillofacial face can be calculated, and the density distribution of the point cloud data of the target maxillofacial face can be matched with the preset maxillofacial face density distribution and the preset mandibular face density distribution to obtain the matching result. For example, for the local preset range of the point cloud data of the target maxillofacial face, the density calculation is performed according to the local preset range, and the density of the point cloud data of the target maxillofacial face within multiple local preset ranges forms the density distribution of the point cloud data of the target maxillofacial face, and the density distribution of the point cloud data of the target maxillofacial face is matched with the preset maxillofacial face density distribution and the preset mandibular face density distribution to obtain a first matching value and a second matching value, and the first matching value and the second matching value are compared, and the maxillofacial face corresponding to the larger matching value is determined as the target maxillofacial face. For example, the first matching value obtained by matching the density distribution of the target maxillary face point cloud data with the preset maxillary face density distribution is 0.65, and the second matching value obtained by matching the density distribution of the target maxillary face point cloud data with the preset mandibular face density distribution is 0.7. Since 0.7 is greater than 0.65, it means that the density distribution of the target maxillary face point cloud data is closer to the preset mandibular face density distribution, and the mandibular face can be determined as the target maxillary face.

[0043] Since factors such as gender and age affect the preset maxillary face density distribution and the preset mandibular face density distribution, the preset maxillary face density distribution and the preset mandibular face density distribution corresponding to different age groups can be set according to age, and the preset maxillary face density distribution is divided into multiple preset maxillary face sub-density distributions, and the preset mandibular face density distribution is divided into multiple preset mandibular face density distributions. The preset maxillary face density distribution and the preset mandibular face density distribution of the corresponding age group are matched according to the age information of the target object, and the density distribution of the point cloud data of the target maxillary face is matched with the preset maxillary face density distribution and the preset mandibular face density distribution of the corresponding age group to obtain a matching result, and the maxillofacial type of the target maxillary face is determined according to the matching result. It is also possible to set the preset maxillary face density distribution and the preset mandibular face density distribution corresponding to different genders according to gender, match the preset maxillary face density distribution and the preset mandibular face density distribution of the corresponding gender according to the gender information of the target object, match the density distribution of the point cloud data of the target maxillary face with the preset maxillary face density distribution and the preset mandibular face density distribution of the corresponding gender, obtain the matching result, and determine the maxillary face type of the target maxillary face according to the matching result. It is also possible to set the preset maxillary face density distribution and preset mandibular face density distribution corresponding to different age groups and different genders according to gender and age, match the preset maxillary face density distribution and preset mandibular face density distribution of the corresponding gender according to the gender information of the target object, set the preset maxillary face density distribution and preset mandibular face density distribution corresponding to different age groups according to age, match the preset maxillary face density distribution and preset mandibular face density distribution of the matched gender according to the age information of the target object, match the preset maxillary face density distribution and preset mandibular face density distribution of the corresponding age group, match the density distribution of the point cloud data of the target maxillary face with the preset maxillary face density distribution and preset mandibular face density distribution of the corresponding age group, respectively, to obtain a matching result, and determine the maxillofacial type of the target maxillary face according to the matching result.

[0044] Optionally, the oral scan image is divided into a maxillary facial scan image and a mandibular facial scan image, including: segmenting the oral scan image to determine the tooth area in the oral scan image; determining multiple bite points based on position information of the tooth area, and forming an occlusal surface based on the multiple bite points; and dividing the oral scan image into a maxillary facial scan image and a mandibular facial scan image based on the occlusal surface.

[0045] Specifically, the position information of the tooth region can be understood as being used to characterize the position of the tooth region in the oral scan image, for example, it can include three-dimensional coordinate information. The occlusal point can be understood as the contact point of the tooth in the occlusal state. The present invention can determine the occlusal point in a static occlusal mode, and can be obtained through the position information of the pixel points in the tooth region. The occlusal surface can be understood as a surface formed by multiple occlusal points, and the occlusal surface can be a plane or a curved surface. The tooth region in the oral scan image can be determined by segmenting the oral scan image through a tooth segmentation model. For example, the oral scan image is input into a pre-trained tooth segmentation model for segmentation processing to obtain the tooth region in the oral scan image, wherein the tooth segmentation model includes but is not limited to a neural network model. The occlusal point can be determined based on the three-dimensional coordinate information of the pixel points in the maxillary tooth region and the three-dimensional coordinate information of the pixel points in the mandibular tooth region. For example, in order to determine whether any pixel point in the maxillary face is an occlusal point, the minimum distance between the pixel point in the maxillary face and all the pixel points in the mandibular face can be determined, and the minimum distance is determined by a preset threshold. If the minimum distance is less than the preset threshold, the pixel point in the maxillary face is determined to be an occlusal point.

[0046] In some embodiments, teeth recognition can be performed on an oral scan image to obtain a tooth recognition result. The tooth recognition result can be an oral scan image with classification marks set. Exemplarily, the classification marks can include background marks, maxillary face marks, and mandibular face marks. Each pixel in the tooth recognition result is respectively set with a classification mark. If the classification mark of a pixel is a background mark, it indicates that the pixel belongs to the background area; if the classification mark of a pixel is a maxillary face mark, it indicates that the pixel belongs to the tooth area, and the maxillofacial type is maxillofacial; if the classification mark of a pixel is a mandibular face mark, it indicates that the pixel belongs to the tooth area, and the maxillofacial type is mandibular. For example, the oral scan image can be processed by a pre-trained tooth recognition model to obtain a tooth recognition result. The tooth recognition model includes but is not limited to a neural network model, such as a convolutional neural network model. For example, the oral scan image is input into the pre-trained tooth recognition model for processing to obtain the tooth recognition result. In the tooth recognition result, the background area is represented by 0, the maxillary face is represented by 1, and the mandibular face is represented by 2. The oral scan image corresponding to 1 in the tooth recognition result is determined as the maxillary face scan image, and the oral scan image corresponding to 2 in the tooth recognition result is determined as the mandibular face scan image. The segmentation plane between the oral scan images corresponding to the upper and lower jaws is determined based on 1 and 2 in the tooth recognition result, and the segmentation plane is determined as the occlusal plane.

[0047] In order to reduce the influence of the background area on the oral scan image, the oral scan image of the background area can be processed, for example, the oral scan image of the background area can be set to the same color, for example, the oral scan image of the background area can be set to black. By extracting the target maxillofacial scan image corresponding to the target maxillofacial in the oral scan image, the amount of registration calculation is reduced, the influence of the remaining maxillofacial on the navigation registration of the surgical robot is reduced, and the accuracy and efficiency of the navigation registration of the surgical robot are improved.

[0048] S130, performing registration based on the target maxillofacial scan image and the target maxillofacial point cloud data to obtain a first transformation matrix.

[0049] In this embodiment, the first conversion matrix can be understood as a matrix that characterizes the conversion relationship between the target maxillofacial scan image and the target maxillofacial point cloud data, which can be obtained by aligning the target maxillofacial scan image and the target maxillofacial point cloud data. The target maxillofacial scan image and the target maxillofacial point cloud data can be aligned through a first alignment model to obtain a first conversion matrix. For example, the target maxillofacial scan image and the target maxillofacial point cloud data can be input into a pre-trained first alignment model for alignment processing to obtain a first conversion matrix. The first alignment model includes but is not limited to a neural network model. Alignment based on the target maxillofacial scan image and the target maxillofacial point cloud data can include multiple iterative alignment processes, and the first conversion matrix is ​​obtained through multiple iterative alignments. Optionally, alignment based on the target maxillofacial scan image and the target maxillofacial point cloud data can include multiple iterative alignment processes, and any iterative alignment process includes a global coarse alignment stage and a point-to-point alignment stage. Among them, the global coarse alignment stage can be understood as alignment performed when the relative position is unknown. The global coarse registration stage can be implemented by the RANSAC algorithm, which can provide data support for the point-to-point registration stage. The point-to-point registration stage can be understood as a precise registration based on the global coarse registration. The point-to-point registration stage can be implemented by the ICP algorithm. The registration is performed through the global coarse registration stage and the point-to-point registration stage, which improves the accuracy and efficiency of the registration.

[0050] During the scanning process of the target maxillofacial face by the oral scanning device, there is an angular offset between the target maxillofacial face scanning image and the point cloud data of the target maxillofacial face, which causes the point cloud data of the target maxillofacial face to be unable to completely match the target maxillofacial face scanning image. Therefore, before aligning the target maxillofacial face scanning image and the point cloud data of the target maxillofacial face, the point cloud data of the target maxillofacial face can be adjusted at an angle to improve the efficiency of the alignment of the target maxillofacial face scanning image and the point cloud data of the target maxillofacial face.

[0051] Optionally, before aligning the target maxillofacial scanning image and the target maxillofacial point cloud data, it also includes: based on the density distribution of the target maxillofacial point cloud data, adjusting the angle of the target maxillofacial point cloud data so that the density distribution of the adjusted target maxillofacial point cloud data meets the gravity distribution condition.

[0052] Specifically, the gravity distribution condition can be understood as a judgment condition for whether the point cloud data of the target maxillofacial face is adjusted in angle. For example, the gravity distribution condition can include that the density distribution of the point cloud data of the target maxillofacial face satisfies the preset density distribution of the point cloud data, and the preset density distribution of the point cloud data can characterize the lateral distribution trend and the longitudinal distribution trend of the point cloud data of the standard maxillofacial face. Taking the maxillofacial face as the target maxillofacial face as an example, the density of the point cloud data of the target maxillofacial face within multiple local preset ranges is calculated respectively, and the density of the point cloud data of the target maxillofacial face within multiple local preset ranges forms the density distribution of the point cloud data of the target maxillofacial face, and the density distribution of the point cloud data of the target maxillofacial face can characterize the lateral distribution trend and the longitudinal distribution trend of the point cloud data of the target maxillofacial face. The density distribution of the point cloud data of the target maxillofacial face is compared with the density distribution of the preset point cloud data. If the density distribution of the point cloud data of the target maxillofacial face is consistent with the density distribution of the preset point cloud data, it means that the density distribution of the point cloud data of the target maxillofacial face satisfies the gravity distribution condition. If the density distribution of the target maxillofacial point cloud data is inconsistent with the density distribution of the preset point cloud data, the lateral and longitudinal angles to be adjusted can be determined according to the angular offset between the lateral and longitudinal distribution trends of the target maxillofacial point cloud data and the lateral and longitudinal distribution trends of the preset point cloud data, and the angle of the target maxillofacial point cloud data is adjusted according to the lateral and longitudinal angles to be adjusted. The density distribution of the target maxillofacial point cloud data after the angle adjustment is compared with the density distribution of the preset point cloud data. If the gravity distribution condition is met, the adjustment is stopped. If the gravity distribution condition is not met, the angle adjustment is continued until the density distribution of the target maxillofacial point cloud data after the angle adjustment meets the gravity distribution condition.

[0053] In some embodiments, the point cloud data of the target maxillofacial region can be angle-adjusted by a point cloud data density distribution model. For example, the point cloud data of the target maxillofacial region and preset point cloud data can be input into a pre-trained point cloud data density distribution model for processing, and the offset angles in the horizontal, vertical and coronal directions can be obtained. The angles to be adjusted in the horizontal, vertical and coronal directions are determined according to the offset angles in the horizontal, vertical and coronal directions. The point cloud data of the target maxillofacial region are angle-adjusted in the horizontal, vertical and coronal directions in turn according to the angles to be adjusted in the horizontal, vertical and coronal directions. The density distribution of the point cloud data of the target maxillofacial region after adjustment satisfies the gravity distribution condition. Before aligning the target maxillofacial scan image and the point cloud data of the target maxillofacial region, the point cloud data of the target maxillofacial region is angle-adjusted, thereby reducing the influence of the offset between the target maxillofacial scan image and the point cloud data of the target maxillofacial region on the alignment of the target maxillofacial scan image and the point cloud data of the target maxillofacial region, and improving the efficiency of the alignment of the target maxillofacial scan image and the point cloud data of the target maxillofacial region.

[0054] S140, extracting the current point cloud data of the auxiliary component from the point cloud data of the target maxillofacial region; performing registration based on the current point cloud data of the auxiliary component and the pre-configured point cloud data of the auxiliary component to obtain a second transformation matrix.

[0055] In this embodiment, the preconfigured point cloud data of the auxiliary component can be understood as the point cloud data of the auxiliary component stored in the surgical robot, and can be pre-stored in the surgical robot before performing the operation. The current point cloud data of the auxiliary component can be extracted from the point cloud data of the target maxillofacial region through a pre-trained model. Optionally, the point cloud data of the target maxillofacial region is input into a pre-trained point cloud classification model to determine the current point cloud data of the auxiliary component. The point cloud classification model includes but is not limited to a neural network model. The second transformation matrix can be understood as a matrix that characterizes the transformation relationship between the current point cloud data of the auxiliary component and the pre-configured point cloud data of the auxiliary component, which can be obtained by aligning the current point cloud data of the auxiliary component and the pre-configured point cloud data of the auxiliary component. For example, the current point cloud data of the auxiliary component and the pre-configured point cloud data of the auxiliary component can be input into a pre-trained second registration model for registration processing to obtain a second transformation matrix. The second registration model includes but is not limited to a neural network model.

[0056] Optionally, the current point cloud data of the auxiliary component is extracted from the point cloud data of the target maxillofacial region, including: matching the point cloud data of the target maxillofacial region with a preset data model of the auxiliary component, and determining the point cloud data corresponding to the preset data model of the auxiliary component as the current point cloud data of the auxiliary component.

[0057] Specifically, the preset data model of the auxiliary component can be understood as a model that characterizes the morphology of the auxiliary component. Optionally, the registration based on the preset data model of the auxiliary component and the point cloud data of the target maxillofacial region can include multiple iterative registration processes, and any iterative registration process includes a global coarse registration stage and a point-to-point registration stage. The registration accuracy can be improved through the two-stage iterative registration process.

[0058] S150: Determine a third conversion matrix based on the first conversion matrix and the second conversion matrix, where the third conversion matrix is ​​used to indicate a conversion relationship between the oral scan image of the target object and the navigation component.

[0059] In this embodiment, the third transformation matrix can be understood as a matrix that characterizes the transformation relationship between the oral scan image of the target object and the navigation component, and can be determined by the first transformation matrix and the second transformation matrix. For example, the third transformation matrix can be obtained by calculating the product between the first transformation matrix and the second transformation matrix.

[0060] The technical solution of this embodiment is to obtain an oral scan image of a target object and point cloud data of a target maxillofacial face, wherein the point cloud data of the target maxillofacial face includes tooth point cloud data and current point cloud data of auxiliary components, and the auxiliary components are used to connect to navigation components; extract a target maxillofacial scan image corresponding to the target maxillofacial face from the oral scan image; perform registration based on the target maxillofacial scan image and the point cloud data of the target maxillofacial face to obtain a first transformation matrix; extract the current point cloud data of the auxiliary components from the point cloud data of the target maxillofacial face; perform registration based on the current point cloud data of the auxiliary components and the pre-configured point cloud data of the auxiliary components to obtain a second transformation matrix; determine a third transformation matrix based on the first transformation matrix and the second transformation matrix, and the third transformation matrix is ​​used to indicate the transformation relationship between the oral scan image of the target object and the navigation component, thereby improving the efficiency and accuracy of surgical robot navigation registration.

[0061] Embodiment 2

[0062] Figure 2 Schematic diagram of the structure of a navigation registration device provided by Embodiment 3 of the present invention. Figure 2 As shown, the device comprises:

[0063] The oral scan image and target maxillofacial point cloud data acquisition module 210 is used to acquire the oral scan image and target maxillofacial point cloud data of the target object, wherein the target maxillofacial point cloud data includes tooth point cloud data and current point cloud data of auxiliary components, and the auxiliary components are used to connect the navigation components;

[0064] A target maxillofacial scan image extraction module 220 is used to extract a target maxillofacial scan image corresponding to the target maxillofacial face from the oral scan image;

[0065] A first conversion matrix determination module 230 is used to perform registration based on the target maxillofacial scan image and the target maxillofacial point cloud data to obtain a first conversion matrix;

[0066] The second transformation matrix determination module 240 is used to extract the current point cloud data of the auxiliary component from the point cloud data of the target maxillofacial region; perform registration based on the current point cloud data of the auxiliary component and the pre-configured point cloud data of the auxiliary component to obtain a second transformation matrix;

[0067] The third conversion matrix determination module 250 is used to determine a third conversion matrix based on the first conversion matrix and the second conversion matrix, where the third conversion matrix is ​​used to indicate the conversion relationship between the oral scan image of the target object and the navigation component.

[0068] Based on the above embodiment, optionally, the target maxillofacial scan image extraction module 220 is further used to: divide the oral scan image into maxillofacial scan image and mandibular scan image; and determine the maxillofacial scan image or the mandibular scan image as the target maxillofacial scan image based on the maxillofacial type.

[0069] Optionally, the target maxillofacial scanning image extraction module 220 is also used to: determine the density distribution of the target maxillofacial point cloud data, match the density distribution of the target maxillofacial point cloud data with the preset maxillofacial density distribution and the preset mandibular density distribution, respectively, to obtain a matching result; determine the maxillofacial type of the target maxillofacial based on the matching result, the maxillofacial type including the maxillofacial and mandibular.

[0070] Optionally, the target maxillofacial scan image extraction module 220 is also used to: segment the oral scan image to determine the tooth area in the oral scan image; determine multiple bite points based on the position information of the tooth area, and form an occlusal surface based on the multiple bite points; and divide the oral scan image into a maxillofacial scan image and a mandibular scan image based on the occlusal surface.

[0071] Optionally, the first transformation matrix determination module 230 is also used to: before aligning the target maxillofacial scanning image and the target maxillofacial point cloud data, adjust the angle of the target maxillofacial point cloud data based on the density distribution of the target maxillofacial point cloud data, so that the density distribution of the adjusted target maxillofacial point cloud data meets the gravity distribution condition.

[0072] Optionally, the second transformation matrix determination module 240 is also used to match the preset data model of the auxiliary component with the point cloud data of the target maxillofacial region, and determine the point cloud data corresponding to the preset data model of the auxiliary component as the current point cloud data of the auxiliary component.

[0073] Optionally, the second transformation matrix determination module 240 is further used to: input the point cloud data of the target maxillofacial region into a pre-trained point cloud classification model to determine the current point cloud data of the auxiliary component.

[0074] Optionally, the registration includes multiple iterative registration processes, and any iterative registration process includes a global coarse registration stage and a point-to-point registration stage.

[0075] The navigation and registration device provided in the embodiment of the present invention can execute the navigation and registration method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method.

[0076] Embodiment 3

[0077] Figure 3 1 is a schematic diagram of the structure of an electronic device provided in Embodiment 3 of the present invention. The electronic device 10 is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices (such as helmets, glasses, watches, etc.) and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present invention described and / or required herein.

[0078] like Figure 3 As shown, the electronic device 10 includes at least one processor 11, and a memory connected to the at least one processor 11, such as a read-only memory (ROM) 12, a random access memory (RAM) 13, etc., wherein the memory stores a computer program that can be executed by at least one processor, and the processor 11 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 12 or the computer program loaded from the storage unit 18 to the random access memory (RAM) 13. In the RAM 13, various programs and data required for the operation of the electronic device 10 can also be stored. The processor 11, the ROM 12, and the RAM 13 are connected to each other through a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0079] A number of components in the electronic device 10 are connected to the I / O interface 15, including: an input unit 16, such as a keyboard, a mouse, etc.; an output unit 17, such as various types of displays, speakers, etc.; a storage unit 18, such as a disk, an optical disk, etc.; and a communication unit 19, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 19 allows the electronic device 10 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0080] The processor 11 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The processor 11 executes the various methods and processes described above, such as the navigation registration method.

[0081] In some embodiments, the navigation registration method may be implemented as a computer program, which is tangibly contained in a computer-readable storage medium, such as a storage unit 18. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 10 via the ROM 12 and / or the communication unit 19. When the computer program is loaded into the RAM 13 and executed by the processor 11, one or more steps of the navigation registration method described above may be performed. Alternatively, in other embodiments, the processor 11 may be configured to perform the navigation registration method in any other appropriate manner (e.g., by means of firmware).

[0082] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0083] The computer program for implementing the navigation registration method of the present invention can be written in any combination of one or more programming languages. These computer programs can be provided to a processor of a general-purpose computer, a special-purpose computer or other programmable data processing device, so that when the computer program is executed by the processor, the functions / operations specified in the flow chart and / or block diagram are implemented. The computer program can be executed entirely on the machine, partially on the machine, partially on the machine as an independent software package and partially on a remote machine, or completely on a remote machine or server.

[0084] Embodiment 4

[0085] Embodiment 4 of the present invention further provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to execute a navigation registration method, the method comprising:

[0086] Acquire the oral scan image of the target object and the point cloud data of the target maxillofacial face, wherein the point cloud data of the target maxillofacial face includes the tooth point cloud data and the current point cloud data of the auxiliary component, and the auxiliary component is used to connect the navigation component; extract the target maxillofacial scan image corresponding to the target maxillofacial face from the oral scan image; perform registration based on the target maxillofacial scan image and the point cloud data of the target maxillofacial face to obtain a first conversion matrix; extract the current point cloud data of the auxiliary component from the point cloud data of the target maxillofacial face; perform registration based on the current point cloud data of the auxiliary component and the pre-configured point cloud data of the auxiliary component to obtain a second conversion matrix; determine a third conversion matrix based on the first conversion matrix and the second conversion matrix, and the third conversion matrix is ​​used to indicate the conversion relationship between the oral scan image of the target object and the navigation component.

[0087] In the context of the present invention, a computer-readable storage medium may be a tangible medium that may contain or store a computer program for use by or in combination with an instruction execution system, device or equipment. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices or equipment, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0088] To provide interaction with a user, the systems and techniques described herein may be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or trackball) through which the user can provide input to the electronic device. Other types of devices may also be used to provide interaction with the user; for example, the feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form (including acoustic input, voice input, or tactile input).

[0089] The systems and techniques described herein may be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.

[0090] A computing system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The client and server relationship is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services.

[0091] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps described in the present invention can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solution of the present invention can be achieved, and this document does not limit this.

[0092] The above specific implementations do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A navigation registration method, characterized in that: include: Acquire an oral scan image of a target object and point cloud data of a target maxillofacial region, wherein the point cloud data of the target maxillofacial region includes tooth point cloud data and current point cloud data of an auxiliary component, wherein the auxiliary component is used to connect to a navigation component; Extracting a target maxillofacial scan image corresponding to the target maxillofacial region from the oral scan image; Performing registration based on the target maxillofacial scan image and the target maxillofacial point cloud data to obtain a first transformation matrix; Extracting the current point cloud data of the auxiliary component from the point cloud data of the target maxillofacial region; performing registration based on the current point cloud data of the auxiliary component and the pre-configured point cloud data of the auxiliary component to obtain a second transformation matrix; A third transformation matrix is ​​determined based on the first transformation matrix and the second transformation matrix, and the third transformation matrix is ​​used to indicate a transformation relationship between the oral scan image of the target object and the navigation component.

2. The method according to claim 1, characterized in that Extracting a target maxillofacial scan image corresponding to the target maxillofacial region from the oral scan image includes: Determining the maxillofacial type of the target maxillofacial region; Dividing the oral scan image into a maxillofacial scan image and a mandibular scan image; The maxillofacial scan image or the mandibular scan image is determined as the target maxillofacial scan image based on the maxillofacial type.

3. The method according to claim 2, characterized in that Determining the maxillofacial type of the target maxillofacial region includes: Determine the density distribution of the point cloud data of the target maxillofacial surface, and match the density distribution of the point cloud data of the target maxillofacial surface with the preset maxillofacial density distribution and the preset mandibular density distribution to obtain a matching result; The maxillofacial type of the target maxillofacial face is determined based on the matching result, where the maxillofacial type includes an upper maxillofacial face and a lower maxillofacial face.

4. The method according to claim 2, characterized in that: The oral scan image is divided into a maxillary face scan image and a mandibular face scan image, including: Segmenting the oral scan image to determine the tooth area in the oral scan image; determining a plurality of occlusal points based on the position information of the tooth region, and forming an occlusal surface based on the plurality of occlusal points; The oral scan image is divided into a maxillary face scan image and a mandibular face scan image based on the occlusal surface.

5. The method according to claim 1, characterized in that Before registering the target maxillofacial scan image and the target maxillofacial point cloud data, the method further includes: Based on the density distribution of the target maxillofacial point cloud data, the target maxillofacial point cloud data is angle-adjusted so that the density distribution of the target maxillofacial point cloud data after adjustment satisfies the gravity distribution condition.

6. The method according to claim 1, characterized in that Extracting the current point cloud data of the auxiliary component from the point cloud data of the target maxillofacial region comprises: Based on matching the preset data model of the auxiliary component with the point cloud data of the target maxillofacial region, the point cloud data corresponding to the preset data model of the auxiliary component is determined as the current point cloud data of the auxiliary component; or, The point cloud data of the target maxillofacial region is input into a pre-trained point cloud classification model to determine the current point cloud data of the auxiliary component.

7. The method according to claim 1, characterized in that The registration includes multiple iterative registration processes, and any iterative registration process includes a global coarse registration stage and a point-to-point registration stage.

8. A navigation registration device, characterized in that: include: An oral scan image and target maxillofacial point cloud data acquisition module, used to acquire an oral scan image of a target object and point cloud data of a target maxillofacial surface, wherein the point cloud data of the target maxillofacial surface includes tooth point cloud data and current point cloud data of an auxiliary component, and the auxiliary component is used to connect to a navigation component; A target maxillofacial scan image extraction module, used for extracting a target maxillofacial scan image corresponding to the target maxillofacial face from the oral scan image; A first conversion matrix determination module, used for performing registration based on the target maxillofacial scan image and the target maxillofacial point cloud data to obtain a first conversion matrix; A second transformation matrix determination module is used to extract the current point cloud data of the auxiliary component from the point cloud data of the target maxillofacial region; perform registration based on the current point cloud data of the auxiliary component and the pre-configured point cloud data of the auxiliary component to obtain a second transformation matrix; The third conversion matrix determination module is used to determine a third conversion matrix based on the first conversion matrix and the second conversion matrix, and the third conversion matrix is ​​used to indicate the conversion relationship between the oral scan image of the target object and the navigation component.

9. An electronic device, characterized in that: The electronic device comprises: at least one processor; and a memory communicatively connected to the at least one processor; wherein, The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can perform the navigation and registration method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the navigation and registration method according to any one of claims 1 to 7 when executed.

Images