Orthodontic bracket bonding guiding method, system and equipment

Abstract

The invention discloses a guiding method, system and equipment for orthodontic bracket bonding. The method comprises the following steps: acquiring an operation scheme of performing orthodontic bracket bonding on a patient by medical personnel; collecting a video stream of the medical personnel executing the operation scheme on the patient in real time; identifying the video stream, and determining real-time operation information currently executed by the medical staff; according to the real-time operation information and the operation scheme, generating a virtual portrait for guiding the medical staff; according to the real-time operation information, image-text information used for guiding the medical staff is generated. Through the generated virtual portrait, the deviation between the current position and the target position of the bracket is visually fed back in real time when the medical staff adheres the bracket, meanwhile, through the generated graphic and text information, the cooperation dependence on medical care is reduced, the medical staff is guided to adhere the bracket to the patient in real time, and the adhering efficiency is improved.

Description

Technical Field

[0001] This invention relates to the field of orthodontic medical assistance technology, and in particular to a method, system and device for guiding the bonding of orthodontic brackets. Background Technology

[0002] Precise bonding of orthodontic brackets is a crucial first step for successful orthodontic treatment. Currently, the "four-handed operation" model is widely used in clinical practice, which requires two medical staff, a doctor and a nurse, to work together to complete the bracket bonding process. This process typically includes: moisture isolation, acid etching, rinsing and drying, applying primer / adhesive, bracket positioning, bonding, light curing, and cleaning up excess material.

[0003] However, the above-mentioned "four-hand operation" bracket bonding process has the following problems: 1. Doctors need to frequently switch their line of sight between the patient's mouth, the bonding tool, and the bracket during the operation, which can easily lead to visual and operational fatigue and errors. Doctors mainly rely on their senses to make fine adjustments, and cannot achieve real-time, visual feedback on the deviation between the current position of the bracket and the target position.

[0004] 2. In the process of positioning and transferring instruments and materials, the cooperation between medical staff is crucial. If the cooperation between doctors and nurses is not smooth, it will lead to a longer bonding time for a single bracket and affect the bonding efficiency.

[0005] Therefore, there is an urgent need for a method that can guide medical staff to perform bracket bonding on patients in real time. Summary of the Invention

[0006] In order to overcome the shortcomings of the prior art, the present invention aims to provide a guiding method, system and equipment for orthodontic bracket bonding, which can guide medical staff to bond brackets to patients in real time, reduce the dependence on cooperation between medical staff and effectively improve bonding efficiency.

[0007] This invention is implemented according to the following scheme: A guiding method for bonding orthodontic brackets is provided, comprising: Obtain the procedure for medical staff to bond orthodontic brackets to patients; Real-time video streaming of the medical staff performing the operation plan on the patient; The video stream is identified to determine the real-time operation information currently being performed by the medical staff; Based on the real-time operation information and the operation plan, a virtual profile is generated to guide the medical staff. Based on the real-time operation information, graphic and textual information is generated to guide the medical staff.

[0008] Compared with the prior art, the beneficial effects of the orthodontic bracket bonding guidance method of the present invention are as follows: by generating a virtual image, the deviation between the current position and the target position of the bracket can be fed back in real time and visually when medical staff bond the bracket. At the same time, the generated graphic information reduces the dependence on cooperation between medical staff and improves bonding efficiency by guiding medical staff to bond the bracket to the patient in real time.

[0009] Optionally, the real-time operation information includes tooth information, tool information, and real-time operation stage; The video stream is identified to determine the real-time operation information currently being performed by the medical staff, including: The video stream is segmented to obtain multiple frames of images; The tooth information and tool information are obtained by identifying the multiple frames of images; The real-time operation phase is determined based on the tool information.

[0010] Optionally, the tooth information includes the three-dimensional poses of multiple teeth in the patient's oral cavity; The patient's dental information is obtained by identifying the multiple frames of images, including: For each frame of the image, identify multiple teeth in the patient's mouth to obtain tooth images corresponding to each tooth; Based on images of multiple teeth, the three-dimensional pose of the teeth is determined.

[0011] Optionally, the tool information includes the positions and six degrees of freedom postures of the multiple tools held by the medical staff. The video stream is identified to obtain the tool information, including: Each frame of the image is segmented, and multiple tools held by the medical staff are identified in each frame of the image to obtain tool images corresponding to each tool. Based on the tool images corresponding to multiple tools, the position and six-degree-of-freedom pose of the tool are determined.

[0012] Optionally, the real-time operation stage is determined based on the tool information, including: Based on the tool information, determine the personnel behavior of the medical staff; Based on the personnel's behavior, determine the current real-time operation phase being performed by the medical staff.

[0013] Optionally, the medical personnel include doctors; Based on the real-time operation information and the operation plan, a virtual profile is generated to guide the doctor, including: Based on the aforementioned operating procedure, determine the ideal coordinates of the bracket corresponding to each tooth; Based on the tooth information and the ideal coordinates of the bracket, determine the bracket positioning coordinates for each tooth used to bond the bracket; Based on the bracket positioning coordinates and the real-time operation information, a virtual image is generated to guide the doctor to attach the bracket to the bracket positioning coordinates.

[0014] Optionally, the medical personnel may also include nurses; Based on the real-time operation information, graphic and textual information is generated to guide the nurse, including: Based on the real-time operation phase, determine the future operation phase to be performed by the nurse; Based on the described future operational phases, determine the operational steps and tools to be used; Based on the steps to be performed and the tools to be used, generate graphic and textual information to guide the nurse in cooperating with the doctor to attach the bracket.

[0015] A guiding system for orthodontic bracket bonding is also provided, which applies the above-mentioned guiding method for orthodontic bracket bonding, including: The visual acquisition module is integrated into the treatment light, the head-mounted device of medical staff, or an independent adjustable bracket, and is used to acquire video streams of medical staff performing the operation plan on patients in real time. The scheme acquisition module is used to acquire the operation scheme of the medical staff for bonding orthodontic brackets to the patient; The data processing module identifies the video stream and determines the real-time operation information currently being performed by the medical staff. The guide information generation module is used for: Based on the real-time operation information and the operation plan, a virtual profile is generated to guide the medical staff. Based on the real-time operation information, generate graphic and textual information to guide the medical staff; The display module is used to display the virtual image and the graphic information.

[0016] Optionally, the medical personnel include doctors and nurses; the display module includes: The AR guidance unit worn by the doctor is used to overlay the virtual image onto the patient's real teeth image; The graphic guidance unit is located on the nurse's side and is used to display the graphic information.

[0017] A computer device is also provided, including a processor and a memory, wherein the memory stores at least one instruction, at least one program, code set or instruction set, wherein the at least one instruction, at least one program, code set or instruction set is loaded and executed by the processor to implement the described orthodontic bracket bonding guidance method. Attached Figure Description

[0018] Figure 1 This is a flowchart of the guiding method of the present invention; Figure 2 This is a schematic diagram illustrating the overlay of a real image onto a virtual image, as presented in this invention. Figure 3 This is a schematic diagram of the graphic information of the present invention. Detailed Implementation

[0019] The preferred embodiments of the present invention will be described below with reference to the accompanying drawings. It should be understood that the preferred embodiments described herein are for illustration and explanation only and are not intended to limit the present invention.

[0020] In the following description, when referring to the accompanying drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims. In the description of this application, it should be understood that the terms "first," "second," "third," etc., are used only to distinguish similar objects and are not necessarily used to describe a specific order or sequence, nor should they be construed as indicating or implying relative importance. Those skilled in the art can understand the specific meaning of the above terms in this application according to the specific circumstances.

[0021] See Figure 1 As shown, a guiding method for orthodontic bracket bonding according to the present invention includes: S1: Obtain the operational plan for orthodontic bracket bonding performed by medical staff on patients. This operational plan is generated by scanning the patient's oral cavity before the orthodontic bracket bonding, obtaining intraoral scan data, and generating a treatment plan based on the intraoral scan data. This plan includes the bracket type corresponding to each tooth, the ideal coordinates of the bracket, the installation angle, and a standardized four-handed operation template. The operation template strictly adheres to the sequence of steps including but not limited to moisture isolation, acid etching, rinsing and drying, applying primer / adhesive, bracket positioning, bonding, light curing, and cleaning of excess material. This invention provides a standardized treatment plan for medical staff by obtaining the operational plan, avoiding differences in process details and rhythm between different medical staff or even the same medical staff in different operations, which could affect the repeatability of orthodontic treatment results and overall quality control.

[0022] S2: Real-time acquisition of video streams of medical staff performing procedures on patients; in this invention, acquisition can be performed by cameras integrated into treatment lights, medical staff's head-mounted devices, or independent adjustable supports to ensure that the video streams acquired by the cameras are complete, thereby ensuring the accuracy of subsequent guidance information generated based on the video streams.

[0023] S3: Recognize the video stream to determine the real-time operation information currently being performed by medical staff; the real-time operation information includes tooth information, tool information, and real-time operation stage. The tooth information includes tooth information of multiple teeth inside the patient's mouth, the tool information includes relevant information of instruments and materials held by medical staff, and the real-time operation stage is used to indicate the steps currently being performed by medical staff.

[0024] In one embodiment of the present invention, identifying the video stream to determine the real-time operation information currently being performed by medical personnel includes: segmenting the video stream to obtain multiple frames of images; identifying the multiple frames of images to obtain tooth information and tool information; and determining the real-time operation stage based on the tool information.

[0025] In one embodiment of the present invention, the tooth information includes the three-dimensional poses of multiple teeth in the patient's oral cavity; the tooth information of the patient is obtained by recognizing multiple frames of images, including: recognizing multiple teeth in the patient's oral cavity for each frame of image to obtain tooth images corresponding to the multiple teeth; and determining the three-dimensional poses of the teeth based on the tooth images of the multiple teeth, wherein the three-dimensional poses of the teeth are used to indicate the spatial position of the teeth inside the patient's oral cavity.

[0026] In one embodiment of the present invention, the tool information includes the positions and six-degree-of-freedom poses of multiple tools held by medical personnel; the tool information is obtained by identifying the video stream, including: segmenting each frame of the image, identifying multiple tools held by medical personnel in each frame of the image, and obtaining tool images corresponding to the multiple tools; and determining the positions and six-degree-of-freedom poses of the tools based on the tool images corresponding to the multiple tools.

[0027] In one embodiment of the present invention, the types of tools held by medical personnel include: instruments and materials, wherein the instruments include, but are not limited to, probes, needle holders, light curing lamps, and saliva suction tubes used in the bonding process, and the materials include, but are not limited to, trays and adhesives for bonding the trays.

[0028] In one embodiment of the present invention, determining the real-time operation stage based on tool information includes: determining the personnel behavior of medical staff based on tool information; and determining the current real-time operation stage performed by medical staff based on personnel behavior.

[0029] This invention accurately infers the behavior of medical personnel by analyzing the changes in tool position and six-degree-of-freedom posture, as well as the usage process of different tools in different steps. For example, the positioning action of the bracket near the tooth corresponds to the bracket positioning step in the bonding process; the irradiation action of the light curing lamp corresponds to the light curing step in the bonding process; and the application action of the adhesive corresponds to the application of the primer / adhesive step in the bonding process. By improving the accuracy of determining personnel behavior, this invention further ensures the accuracy of identifying real-time operation stages.

[0030] Meanwhile, this invention achieves dynamic tracking and status clarity of instruments and materials held by medical staff through the corresponding position and six-degree-of-freedom posture of the tool, avoiding the inability to identify whether the tool held by the medical staff is in an unused or used state by only recognizing the tool type; and by clarifying the real-time spatial state of the bracket during the operation through the position and six-degree-of-freedom posture of the bracket, it ensures the accuracy of the subsequent generation of information to guide medical staff in aligning the bracket with the teeth, thereby ensuring the accuracy of the bracket bonding in the spatial dimension of the teeth.

[0031] S4: Generate a virtual profile to guide medical staff based on real-time operation information and operation plan; In one embodiment of the present invention, the medical personnel include doctors; based on real-time operation information and operation plan, a virtual image for guiding the doctor is generated, including: determining the ideal bracket coordinates corresponding to each tooth according to the operation plan; determining the bracket positioning coordinates for each tooth for bracket bonding based on tooth information and ideal bracket coordinates; and generating a virtual image for guiding the doctor to bond the brackets to the bracket positioning coordinates based on the bracket positioning coordinates and real-time operation information; see also Figure 2 The diagram illustrates a virtual image overlaid on a real image, viewed by the doctor. This virtual image is displayed through AR-enabled glasses worn by the doctor. When the doctor wears the glasses, the virtual image is overlaid on the patient's real image, providing real-time, visual feedback on the deviation between the current position and the target position of the bracket, guiding the doctor to accurately perform the bonding operation.

[0032] S5: Generate graphic and textual information to guide medical staff based on real-time operation information.

[0033] In one embodiment of the present invention, the medical personnel also include a nurse, who assists the doctor in completing the bonding process by passing tools and providing prompts for the next operation. Based on real-time operation information, graphic and textual information is generated to guide the nurse, including: determining the future operation stage to be performed by the nurse based on the real-time operation stage; determining the operation steps and tools to be used based on the future operation stage; and generating graphic and textual information to guide the nurse in cooperating with the doctor to bond the bracket based on the operation steps and tools to be used. See also... Figure 3As shown, this diagram illustrates the graphic information viewed by nurses. By viewing this information, nurses can quickly identify the tools needed by doctors for future procedures and provide guidance for those procedures. For example, the graphic information includes diagrams of tools to be handed over and textual prompts for the doctor's next steps. Nurses can quickly and accurately pass on tools and communicate the next steps, assisting doctors in moving to the next stage and reducing reliance on coordination between medical staff.

[0034] The present invention provides a guiding system for orthodontic bracket bonding, which applies the above-mentioned guiding method for orthodontic bracket bonding, comprising: The visual acquisition module is integrated into the treatment light, the head-mounted device of medical staff, or an independent adjustable bracket. It is used to acquire video streams of medical staff performing operation plans on patients in real time. The video stream includes the hands of medical staff and the inside of the patient's mouth to extract information on the tools held by medical staff, the current real-time operation stage, and the teeth information inside the patient's mouth. The scheme acquisition module is used to acquire the operation scheme of medical staff for bonding orthodontic brackets to patients; The data processing module identifies the video stream and determines the real-time operation information currently being performed by medical staff; The guide information generation module is used for: Based on real-time operation information and operation plans, a virtual profile is generated to guide medical staff. Based on real-time operational information, generate graphic and textual information to guide medical staff; The display module is used to display virtual images and graphic information.

[0035] In one embodiment of the present invention, medical personnel include doctors and nurses; the display module includes: The AR guidance unit worn by the dentist overlays a virtual image onto a real image of the patient's teeth, providing real-time, visual feedback on the deviation between the current and target positions of the brackets. This guides the dentist to accurately perform the bonding procedure. See [link / reference]. Figure 2 The image shown is the display screen after the doctor wears the AR guidance unit; The graphic guidance unit, located on the nurse's side, displays graphic information to allow nurses to clearly view it and quickly and accurately assist doctors in completing bracket bonding. (See also...) Figure 3 The image shown is the screen as seen by the nurse.

[0036] The computer device of the present invention includes a processor and a memory. The memory stores at least one instruction, at least one program, code set, or instruction set, wherein the at least one instruction, at least one program, code set, or instruction set is loaded and executed by the processor to implement the above-described boot method.

[0037] The processor can be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc.

[0038] The memory can be used to store the computer program or module. The processor implements various functions of the boot method by running or executing the computer program or module stored in the memory and calling data stored in the memory. The memory may mainly include a program storage area and a data storage area. The program storage area may store the operating system, applications required for at least one function, etc.; the data storage area may store data created based on the use of the mobile phone, etc. In addition, the memory may include high-speed random access memory, and may also include non-volatile memory, such as hard disk, RAM, plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, at least one disk storage device, flash memory device, or other volatile solid-state storage device.

[0039] The above are merely preferred embodiments of this application and are not intended to limit this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

Claims

1. A method for guiding the bonding of orthodontic brackets, characterized in that, include: Obtain the procedure for medical staff to bond orthodontic brackets to patients; Real-time video streaming of the medical staff performing the operation plan on the patient; The video stream is identified to determine the real-time operation information currently being performed by the medical staff; Based on the real-time operation information and the operation plan, a virtual profile is generated to guide the medical staff. Based on the real-time operation information, graphic and textual information is generated to guide the medical staff.

2. The guiding method for bonding orthodontic brackets according to claim 1, characterized in that, The real-time operation information includes tooth information, tool information, and the real-time operation stage; The video stream is identified to determine the real-time operation information currently being performed by the medical staff, including: The video stream is segmented to obtain multiple frames of images; The tooth information and tool information are obtained by identifying the multiple frames of images; The real-time operation phase is determined based on the tool information.

3. The guiding method for bonding orthodontic brackets according to claim 2, characterized in that, The dental information includes the three-dimensional poses of multiple teeth in the patient's oral cavity. The patient's dental information is obtained by identifying the multiple frames of images, including: For each frame of the image, identify multiple teeth in the patient's mouth to obtain tooth images corresponding to each tooth; Based on images of multiple teeth, the three-dimensional pose of the teeth is determined.

4. The guiding method for bonding orthodontic brackets according to claim 3, characterized in that, The tool information includes the positions and six degrees of freedom postures of the multiple tools held by the medical staff. The video stream is identified to obtain the tool information, including: Each frame of the image is segmented, and multiple tools held by the medical staff are identified in each frame of the image to obtain tool images corresponding to each tool. Based on the tool images corresponding to multiple tools, the position and six-degree-of-freedom pose of the tool are determined.

5. The guiding method for bonding orthodontic brackets according to claim 2, characterized in that, Based on the tool information, the real-time operation phase is determined, including: Based on the tool information, determine the personnel behavior of the medical staff; Based on the personnel's behavior, determine the current real-time operation phase being performed by the medical staff.

6. The guiding method for bonding orthodontic brackets according to claim 4, characterized in that, The medical staff include doctors; Based on the real-time operation information and the operation plan, a virtual profile is generated to guide the doctor, including: Based on the aforementioned operating procedure, determine the ideal coordinates of the bracket corresponding to each tooth; Based on the tooth information and the ideal coordinates of the bracket, determine the bracket positioning coordinates for each tooth used to bond the bracket; Based on the bracket positioning coordinates and the real-time operation information, a virtual image is generated to guide the doctor to attach the bracket to the bracket positioning coordinates.

7. The guiding method for bonding orthodontic brackets according to claim 6, characterized in that, The medical staff also include nurses; Based on the real-time operation information, graphic and textual information is generated to guide the nurse, including: Based on the real-time operation phase, determine the future operation phase to be performed by the nurse; Based on the described future operational phases, determine the operational steps and tools to be used; Based on the steps to be performed and the tools to be used, generate graphic and textual information to guide the nurse in cooperating with the doctor to attach the bracket.

8. A guiding system for orthodontic bracket bonding, employing the guiding method for orthodontic bracket bonding as described in any one of claims 1-7, characterized in that, include: The visual acquisition module is integrated into the treatment light, the head-mounted device of medical staff, or an independent adjustable bracket, and is used to acquire video streams of medical staff performing the operation plan on patients in real time. The scheme acquisition module is used to acquire the operation scheme of the medical staff for bonding orthodontic brackets to the patient; The data processing module identifies the video stream and determines the real-time operation information currently being performed by the medical staff. The guide information generation module is used for: Based on the real-time operation information and the operation plan, a virtual profile is generated to guide the medical staff. Based on the real-time operation information, generate graphic and textual information to guide the medical staff; The display module is used to display the virtual image and the graphic information.

9. The guiding system for orthodontic bracket bonding according to claim 8, characterized in that, The medical staff include doctors and nurses; The display module includes: The AR guidance unit worn by the doctor is used to overlay the virtual image onto the patient's real teeth image; The graphic guidance unit is located on the nurse's side and is used to display the graphic information.

10. A computer device, characterized in that, The computer device includes a processor and a memory, the memory storing at least one instruction, at least one program, code set, or instruction set, the at least one instruction, at least one program, code set, or instruction set being loaded and executed by the processor to implement a guiding method for orthodontic bracket bonding as described in any one of claims 1 to 7.

Images