Intraoral suture key element detection method based on deep learning

Abstract

The invention discloses an intraoral suturing key element detection method based on deep learning, and relates to the technical field of gingival incision suturing. The invention discloses an intraoral suture key element detection method based on deep learning. The method comprises the following steps: inputting gingival zone texture point cloud; calculating a normal vector of each point in the point cloud; the method comprises the following steps: acquiring incision information, end points and each middle point on the gingiva, and carrying out statistical average calculation on an optimal projection direction according to the length and position of the incision on the gingiva and the normal vector of each point; according to the method, on the basis of a deep learning method, the features such as textures, depths and normal vectors of scanning model point clouds are utilized for coding, instance segmentation, suture instance extraction and skeletization can be utilized on multiple projection results, a detection result of suture features is obtained, manual suture result identification is replaced with a digital intelligence means, and the method is efficient and objective. One-to-one guidance and analysis of operation results can be realized, and students can be helped to realize autonomous learning. And data can be recorded, so that operation problems can be counted and analyzed, and targeted teaching is realized.

Description

Technical Field

[0001] This invention relates to the field of gingival incision suturing technology, specifically to a method for detecting key elements of intraoral suturing based on deep learning. Background Technology

[0002] Intraoral suturing is a crucial technique for wound closure in oral surgery. To better promote wound healing, reduce infection, and achieve aesthetic and functional recovery, every oral clinician must master the relevant suturing instruments, suturing methods, and key operational points, and be able to implement the optimal suturing method according to the requirements of clinical cases. However, in teaching intraoral suturing, practical instruction currently mainly uses suturing models. After students complete their practice, the instructor visually assesses whether the incision, suture method, stitch length, edge distance, knot, and suture end meet the requirements. This method has the following main problems: 1. Inefficient and highly subjective; 2. Due to insufficient teacher-student ratio, students cannot receive timely guidance on operational issues; 3. Data was not retained, making it impossible to analyze common student problems and hindering targeted teaching. Summary of the Invention

[0003] The purpose of this invention is to provide a method for detecting key elements of intraoral suturing based on deep learning, which solves the problems of low efficiency and high subjectivity, insufficient teacher-student ratio, inability to provide timely guidance for students' operational problems, lack of data retention, inability to analyze common student problems, and difficulty in achieving targeted teaching.

[0004] To achieve the above objectives, the present invention provides the following technical solution: A deep learning-based method for detecting key elements of intraoral suturing includes the following steps: S1: Input gingival texture point cloud; S2: Calculate the normal vector of each point in the point cloud; S3: Obtain information about the incision on the gum, including the endpoints and intermediate points; S4: Calculate the optimal projection direction by statistically averaging the length and location of the incision on the gum line and the normal vectors of each point; Based on a pre-set template point cloud, a new point cloud is matched to obtain a standard projection direction. The specific method is to pre-train the template point cloud and obtain point pair features. Let the two points be m1 and m2, then the characteristics of the point pair are: The normal vectors of the two points are n1 and n2, and the distance vector is d = m2 - m1. The point pair features in the template point cloud are used to form a hash table to facilitate subsequent matching; Matching is achieved through voting or an encoder network to obtain the final coarse matching result output, while simultaneously optimizing the result using a fine point cloud matching method. Template point cloud: , Target point cloud: , To minimize the sum of squared errors, we obtain the matching R,t: , S5: Using the projection direction calculated in the previous step as the center, project uniformly in all directions within a certain viewing cone angle to obtain texture maps and depth maps in each direction. The diameter of the projected sphere is fixed. by The interval of degrees for Euler angles Three angles are sampled, and Euler angles are converted into a rotation matrix R. , The coordinates of a point in a point cloud are: ; Therefore, the formulas for projecting onto the 2D texture map and depth map are: ; S6: Combine the three color channels of the texture map with the depth information of the depth map to form a new 4-channel image that contains both color and depth information, used to determine the position of stitches, thread ends, and knots. S7: Detect the locations of sutures, thread ends, and knots using image instance segmentation and object detection algorithms. The loss function of the object detection and instance segmentation detection network is designed as follows: ; Classification loss; Bounding box loss; Binary mask loss; S8: Perform final fusion of multi-angle image detection information, use the maximum suppression algorithm to remove overlapping detections, summarize the final detection results, select the detection box with the highest score, and calculate the Intersection over Union (IOU). ; S9: Determine the type of suture based on the cut information, thread end points, and knot information; S10: Based on the suture type results, determine the overall suture type and generate relevant information such as the distance from the suture end to the incision and the spacing between suture ends.

[0005] Furthermore, in S6, data from the green channel is removed and replaced with data from the deep channel.

[0006] Furthermore, the types of sutures in S9 include vertical, horizontal, and interrupted types, where the suture end is parallel to the incision and the suture end crosses the incision. Vertical means the suture end is perpendicular to the incision, horizontal means the suture end is parallel to the incision, and interrupted means the suture end crosses the incision.

[0007] The beneficial effects of this invention are as follows: This application utilizes deep learning methods to encode features such as texture, depth, and normal vectors of scanned model point clouds. Based on multiple preset suture model point cloud templates, it quickly matches and obtains the model type and the centering direction of the model's frontal orientation, preparing for subsequent suture feature extraction algorithms. Simultaneously, the textured, center-facing suture model point cloud is projected onto multiple preset projection directions. Instance segmentation and target detection algorithms are used to extract suture instances from the multiple projection results, which are then skeletonized to obtain suture feature detection results. Finally, the detection results from multiple projection directions are fused, labeled back into the model point cloud, and the corresponding suture detection results are calculated. This digital approach replaces manual identification of suture results, making it more efficient and objective. It provides one-on-one guidance and analysis of operational results, helping students achieve self-directed learning. Data can be recorded, and operational problems can be statistically analyzed to achieve targeted teaching.

[0008] The above description is merely an overview of the technical solution of the present invention. In order to better understand the technical means of the present invention and to implement it in accordance with the contents of the specification, the preferred embodiments of the present invention are described in detail below with reference to the accompanying drawings. Attached Figure Description

[0009] Figure 1 This is a flowchart illustrating the model recognition process according to an embodiment of the present invention.

[0010] Figure 2 This is a schematic diagram of a projection method according to an embodiment of the present invention.

[0011] Figure 3 This is a schematic diagram of the model recognition result according to an embodiment of the present invention. Detailed Implementation

[0012] The technical solution of the present invention will now be clearly and completely described with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0013] In the description of this invention, it should be noted that the terms "center," "upper," "lower," "left," "right," "vertical," "horizontal," "inner," and "outer," etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are used only for the convenience of describing the invention and for simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on the invention. Furthermore, the terms "first," "second," and "third" are used for descriptive purposes only and should not be construed as indicating or implying relative importance.

[0014] In the description of this invention, it should be noted that, unless otherwise explicitly specified and limited, the terms "installation," "connection," and "linking" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; and they can refer to the internal connection of two components. Those skilled in the art can understand the specific meaning of the above terms in this invention based on the specific circumstances.

[0015] Furthermore, the technical features involved in the different embodiments of the present invention described below can be combined with each other as long as they do not conflict with each other.

[0016] Please see Figure 1 A preferred embodiment of this application illustrates a method for detecting key elements of intraoral suturing based on deep learning, comprising the following steps: S1: Input gingival texture point cloud; S2: Calculate the normal vector of each point in the point cloud; S3: Obtain information about the incision on the gum, including the endpoints and intermediate points; S4: Calculate the optimal projection direction by statistically averaging the length and location of the incision on the gum line and the normal vectors of each point; Based on a pre-set template point cloud, a new point cloud is matched to obtain a standard projection direction. The specific method is to pre-train the template point cloud and obtain point pair features. Let the two points be m1 and m2, then the characteristics of the point pair are: The normal vectors of the two points are n1 and n2, and the distance vector is d = m2 - m1. The point pair features in the template point cloud are used to form a hash table to facilitate subsequent matching; Matching is achieved through voting or an encoder network to obtain the final coarse matching result output, while simultaneously optimizing the result using a fine point cloud matching method. Template point cloud: , Target point cloud: , To minimize the sum of squared errors, we obtain the matching R,t: , S5: Using the projection direction calculated in the previous step as the center, project uniformly in all directions within a certain viewing cone angle to obtain texture maps and depth maps in each direction. The diameter of the projected sphere is fixed. by The interval of degrees for Euler angles Three angles are sampled, and Euler angles are converted into a rotation matrix R. , The coordinates of a point in a point cloud are: ; Therefore, the formulas for projecting onto the 2D texture map and depth map are: ; S6: Combine the three color channels of the texture map with the depth information of the depth map to form a new 4-channel image that contains both color and depth information, used to determine the position of stitches, thread ends, and knots. S7: Detect the locations of sutures, thread ends, and knots using image instance segmentation and object detection algorithms. The loss function of the object detection and instance segmentation detection network is designed as follows: ; Classification loss; Bounding box loss; Binary mask loss; S8: Perform final fusion of multi-angle image detection information, use the maximum suppression algorithm to remove overlapping detections, summarize the final detection results, select the detection box with the highest score, and calculate the Intersection over Union (IOU). ; S9: Determine the type of suture based on the cut information, thread end points, and knot information; S10: Based on the suture type results, determine the overall suture type and generate relevant information such as the distance from the suture end to the incision and the spacing between suture ends.

[0017] In S6, the data from the green channel is removed and replaced with the deep channel.

[0018] The types of sutures in S9 include vertical, horizontal (with the suture end parallel to the incision), and interrupted (with the suture end crossing the incision). Vertical means the suture end is perpendicular to the incision, horizontal means the suture end is parallel to the incision, and interrupted means the suture end crosses the incision.

[0019] In summary, this invention provides a deep learning-based method for detecting key elements of intraoral sutures. This method utilizes deep learning to encode features such as texture, depth, and normal vectors from scanned model point clouds. Based on multiple preset suture model point cloud templates, it quickly matches the model type and locates the model's frontal center orientation for subsequent suture feature extraction algorithms. Simultaneously, the textured, centered suture model point cloud is projected onto multiple preset projection directions. Instance segmentation and target detection algorithms are used to extract suture instances from the multiple projection results, which are then skeletonized to obtain suture feature detection results. Finally, the detection results from multiple projection directions are fused, annotated back into the model point cloud, and the corresponding suture detection results are calculated. This digital approach replaces manual identification of suture results, making it more efficient and objective. It allows for one-on-one guidance and analysis of operational results, helping students achieve self-directed learning. Data can be recorded, and operational problems can be statistically analyzed for targeted teaching.

[0020] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0021] The embodiments described above are merely illustrative of implementation methods of the present invention, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of the invention patent. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and these all fall within the protection scope of the present invention. Therefore, the protection scope of this invention patent should be determined by the appended claims.

Claims

1. A method for detecting key elements of intraoral sutures based on deep learning, characterized in that, Includes the following steps: S1: Input gingival texture point cloud; S2: Calculate the normal vector of each point in the point cloud; S3: Obtain information about the incision on the gum, including the endpoints and intermediate points; S4: Calculate the optimal projection direction by statistically averaging the length and location of the incision on the gum line and the normal vectors of each point; Based on a pre-set template point cloud, a new point cloud is matched to obtain a standard projection direction. The specific method is to pre-train the template point cloud and obtain point pair features. Let the two points be m1 and m2, then the characteristics of the point pair are: The normal vectors of the two points are n1 and n2, and the distance vector is d = m2 - m1. The point pair features in the template point cloud are used to form a hash table to facilitate subsequent matching; Matching is achieved through voting or an encoder network to obtain the final coarse matching result output, while simultaneously optimizing the result using a fine point cloud matching method. Template point cloud: , Target point cloud: , To minimize the sum of squared errors, we obtain the matching R,t: ， S5: Using the projection direction calculated in the previous step as the center, project uniformly in all directions within a certain viewing cone angle to obtain texture maps and depth maps in each direction. The diameter of the projected sphere is fixed. by The interval of degrees for Euler angles Three angles are sampled, and Euler angles are converted into a rotation matrix R. ， The coordinates of a point in a point cloud are: ; Therefore, the formulas for projecting onto the 2D texture map and depth map are: ； S6: Combine the three color channels of the texture map with the depth information of the depth map to form a new 4-channel image that contains both color and depth information, used to determine the position of stitches, thread ends, and knots. S7: Detect the location of sutures, thread ends, and knots using image instance segmentation and object detection algorithms. The loss function of the object detection and instance segmentation detection network is designed as follows: ; Classification loss; Bounding box loss; Binary mask loss; S8: Perform final fusion of multi-angle image detection information, use the maximum suppression algorithm to remove overlapping detections, summarize the final detection results, select the detection box with the highest score, and calculate the Intersection over Union (IOU). ; S9: Determine the type of suture based on the cut information, thread end points, and knot information; S10: Based on the suture type results, determine the overall suture type and generate relevant information such as the distance from the suture end to the incision and the spacing between suture ends.

2. The method for detecting key elements of intraoral sutures based on deep learning as described in claim 1, characterized in that, In S6, the data from the green channel is removed and replaced with the deep channel.

3. The method for detecting key elements of intraoral sutures based on deep learning as described in claim 1, characterized in that, The types of sutures in S9 include vertical, horizontal (with the suture end parallel to the incision), and interrupted (with the suture end crossing the incision). Vertical means the suture end is perpendicular to the incision, horizontal means the suture end is parallel to the incision, and interrupted means the suture end crosses the incision.

Images