An interactive outpatient oral medical record display device and method

Abstract

The present invention belongs to the field of medical information, and specifically relates to an interactive outpatient oral medical record display device and method. The device of the present invention includes: an image acquisition unit, which acquires an oral image, wherein the oral image is a full-mouth tooth position curved surface layer image; a mark acquisition unit, which acquires mark information associated with the oral image, wherein the mark information includes focus point information and reference dental arch information; a spacing information unit, which obtains spacing information according to the oral image and tooth position information; an information drawing unit, which determines the tooth area to be rendered in the canvas according to the mark information, adjusts the tooth model corresponding to the tooth position information in the tooth area to be rendered based on the reference dental arch information, determines the spacing of the tooth model based on the spacing information, draws a patch on the tooth model corresponding to the tooth position information, and draws the focus area patch in the canvas based on the rendering rule corresponding to the focus point information. The present invention can realize the analysis of DICOM, STL, and JPG images, and generate images suitable for medical record display.

Description

Technical Field

[0001] The invention belongs to the field of medical information, and in particular relates to an interactive outpatient oral medical record display device and method. Background Art

[0002] Existing electronic medical record systems are mostly guided by clinical diagnosis, and then introduce relevant text information content. Taking the dental clinic as an example, insufficient oral hygiene leads to the prevalence of dental diseases. In the process of diagnosing oral diseases, dental X-rays are generally taken first, and then the diagnosis is made based on the X-rays. Clinical physicians lack intuitiveness during clinical use and need to perform a lot of text entry work. At the same time, the collection of clinically related images and image information cannot be synchronized to the patient's electronic medical record system in one application interface. Multiple applications need to be opened for selection and insertion (paste), which greatly takes up clinical diagnosis and treatment time. At the same time, it is extremely unfriendly to senior physicians who are not familiar with the use of electronic products. As a result, electronic medical records are all text information. Patients often cannot intuitively see the relevant images and image materials when they get the electronic medical records, thereby reducing communication efficiency. Summary of the invention

[0003] The main purpose of the present invention is to overcome the above-mentioned defects in the prior art and to provide an interactive outpatient oral medical record display device that can make full use of existing medical imaging resources and improve the communication efficiency between doctors and patients and the accuracy of information transmission.

[0004] According to one embodiment of the present invention, an interactive outpatient oral medical record display device includes:

[0005] An image acquisition unit, used for acquiring an oral cavity image, wherein the oral cavity image is a full-mouth tooth position curved surface layer image and contains tooth position information of at least one tooth;

[0006] A marking acquisition unit, used for acquiring marking information associated with the oral image, wherein the marking information includes focus point information and reference dental arch information;

[0007] A spacing information unit, used to obtain spacing information according to the oral image and tooth position information;

[0008] An information drawing unit is used to determine the tooth area to be rendered in the canvas according to the marking information, adjust the tooth model in the tooth area to be rendered and corresponding to the tooth position information based on the reference dental arch information, determine the spacing of the tooth model based on the spacing information, draw a patch on the tooth model corresponding to the tooth position information, and draw the focus area patch in the canvas based on the rendering rule corresponding to the focus point information.

[0009] According to an embodiment of the present invention, the process of acquiring the marking information includes:

[0010] Obtaining feature points associated with the oral image based on tooth position information;

[0011] Determine the peripheral image area in the oral image based on the tooth position information, and determine the inflammation reference information associated with the tooth root based on the grayscale comparison method;

[0012] The reference dental arch information is determined according to the dental pulp distribution characteristics in the oral image.

[0013] According to one embodiment of the present invention, the associated characteristic points include the top position of the tooth, the root position and the size characteristics of the dental pulp tissue;

[0014] The feature points associated with the oral image obtained according to the tooth position information include:

[0015] Determine a processing base point of the oral image according to the marked tooth position information, wherein the processing base point includes an image area corresponding to at least one tooth;

[0016] Determining tooth edges in an oral image based on a grayscale histogram;

[0017] Determining the position of the top of the tooth and the position of the root of the tooth based on the shape defined by the edge of the tooth;

[0018] The size characteristics of the dental pulp tissue corresponding to the tooth are determined based on the shape defined by the tooth edge.

[0019] According to one embodiment of the present invention, determining the tooth edge in the oral image based on the grayscale histogram includes:

[0020] According to the processing base point in the oral image, a reference grayscale histogram at the processing base point is obtained;

[0021] According to a reference grayscale histogram and an oral image within a predetermined area, an edge area corresponding to the oral image is determined, wherein the predetermined area is determined based on a processing base point and an extension direction of teeth in the oral image, and in the oral image, the lower jaw to the upper jaw is a first direction, and the upper jaw to the lower jaw is a second direction, and the extension direction includes an arrangement direction from the processing base point to the molar teeth.

[0022] According to an embodiment of the present invention, the step of determining the edge area corresponding to the oral cavity image includes:

[0023] Segmenting the oral cavity image within a predetermined area to obtain a plurality of segmented sub-images, wherein the sub-images have a pixel size of not less than 10*10;

[0024] Obtaining grayscale histograms of several segmented sub-images respectively, determining edge sub-images for edge region division based on the approximation between the grayscale histogram of each sub-image and the reference grayscale histogram, and determining a closed curve based on the image center of the edge sub-image to obtain a tooth region;

[0025] The margins of individual teeth are obtained according to the tooth regions.

[0026] According to an embodiment of the present invention, the approximation is obtained by a cosine approximation algorithm.

[0027] According to one embodiment of the present invention, the processing base point is the central incisor of the upper or lower jaw.

[0028] According to an embodiment of the present invention, determining the reference dental arch information according to the dental pulp distribution characteristics in the oral image includes:

[0029] obtaining an image of the interior of the tooth edge based on the tooth edge;

[0030] Determine the pulp distribution area inside each tooth based on the pulp grayscale threshold;

[0031] Determine the distribution characteristics of the dental pulp inside each tooth according to the coordinates of the center point of the dental pulp distribution area inside each tooth;

[0032] The characteristics of the dental arch are determined based on the distribution characteristics of the dental pulp.

[0033] According to an embodiment of the present invention, the process of obtaining inflammation reference information includes:

[0034] Acquire tooth root images based on oral CT images;

[0035] Binarize the tooth root image;

[0036] The apical periodontitis detection network is used to identify the tooth root image, and the recognition result of the tooth root image is obtained as the inflammation reference information.

[0037] According to one embodiment of the present invention, there is also provided an interactive outpatient oral medical record display method, comprising:

[0038] Acquire an oral cavity image, wherein the oral cavity image is a full-mouth tooth position curved surface layer image and contains tooth position information of at least one tooth;

[0039] Acquire marking information associated with the oral image, wherein the marking information includes focus point information and reference dental arch information;

[0040] Obtain spacing information based on oral images and tooth position information;

[0041] The tooth area to be rendered in the canvas is determined according to the marking information, the tooth model in the tooth area to be rendered and corresponding to the tooth position information is adjusted based on the reference dental arch information, the spacing of the tooth model is determined based on the spacing information, the patch is drawn on the tooth model corresponding to the tooth position information, and the focus area patch is drawn in the canvas based on the rendering rule corresponding to the focus point information.

[0042] The present invention can improve the efficiency of oral communication and optimize the operation experience of doctors. The present invention can analyze DICOM, STL and JPG images and generate images suitable for medical record display. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] Figure 1 , a schematic diagram of an interactive outpatient oral medical record display device in one embodiment of the present invention;

[0044] Figure 2 , a schematic diagram of a method for obtaining characteristic points associated with an oral image based on tooth position information in one embodiment of the present invention;

[0045] Figure 3 , a schematic diagram of an interactive outpatient oral medical record display method in one embodiment of the present invention. DETAILED DESCRIPTION

[0046] The technical solution in the present application will be described below in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments.

[0047] In the following description, specific details such as specific system structures, technologies, etc. are provided for the purpose of illustration rather than limitation, so as to provide a thorough understanding of the embodiments of the present application. However, it should be clear to those skilled in the art that the present application may also be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to prevent unnecessary details from obstructing the description of the present application.

[0048] The term "include" herein indicates the presence of the described features, integral bodies, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integral bodies, steps, operations, elements, components and / or their collections. The terms "include", "comprising", "having" and their variations all mean "including but not limited to", unless otherwise specifically emphasized.

[0049] In the following, the terms "first" and "second" are used for descriptive purposes only and are not to be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the features. In the description of the embodiments of the present application, unless otherwise specified, "plurality" means two or more.

[0050] The term "and / or" in this article is only a description of the association relationship of the associated objects, indicating that there can be three relationships. For example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this article generally indicates that the associated objects before and after are in an "or" relationship.

[0051] Please refer to Figure 1 According to one embodiment of the present invention, an interactive outpatient oral medical record display device includes:

[0052] An image acquisition unit, used for acquiring an oral cavity image, wherein the oral cavity image is a full-mouth tooth position curved surface layer image and contains tooth position information of at least one tooth;

[0053] A marking acquisition unit, used for acquiring marking information associated with the oral image, wherein the marking information includes focus point information and reference dental arch information;

[0054] A spacing information unit, used to obtain spacing information according to the oral image and tooth position information;

[0055] An information drawing unit is used to determine the tooth area to be rendered in the canvas according to the marking information, adjust the tooth model in the tooth area to be rendered and corresponding to the tooth position information based on the reference dental arch information, determine the spacing of the tooth model based on the spacing information, draw a patch on the tooth model corresponding to the tooth position information, and draw the focus area patch in the canvas based on the rendering rule corresponding to the focus point information.

[0056] This method can reduce the need for information processing based on oral images, make full use of image resources, and improve the efficiency of communication between doctors and patients.

[0057] The image acquisition unit of the present invention is used to acquire oral images, wherein the oral images are generally stored in a database or a designated memory within the hospital after acquisition, and image information corresponding to the user's medical record number or identification code can be obtained through an interface or address;

[0058] The oral image is a full-mouth dental surface layer image, which is also called a dental panoramic film. It uses X-ray fluoroscopy technology to display a two-dimensional plane image. It can display a dental panoramic film, mainly showing teeth, alveolar bones, maxillary and mandibular bones and part of the maxillofacial soft tissue structure, and is suitable for routine examinations of teeth, alveolar bones, maxillary and mandibular bones, such as caries, periodontal disease, pulp disease, alveolar bone lesions, etc. It should be noted that although CBCT or other CT has the function of three-dimensional reconstruction, when displaying the corresponding dental position map, the analysis and reconstruction of the image requires more data support, which increases the complexity of the model. Therefore, the analysis is performed on the basis of the full-mouth dental surface layer image and the information is displayed.

[0059] When processing images, you need to perform preprocessing operations according to different file formats. For example, when the image is in DiCOM format, you can use pydicom or dcm4che to export it as jpg and process it; when the object is STL, you can use JSTL to export it as jpg and process it. When the source data to be processed contains multiple images, you can browse to determine the image to be processed and perform further processing after exporting.

[0060] When acquiring full-mouth dental surface layer images, specific teeth in the image can be marked to determine the basis for subsequent image analysis, reducing the cost of analysis. Theoretically, this step can be omitted by marking and learning a large amount of data, but an additional tooth position determination process needs to be introduced. If the basis for analysis is determined at the beginning, such as selecting incisors as the basis for analysis, that is, marking incisors in the oral image, the order of other teeth is also relatively determined, and the corresponding images are segmented and identified in a certain way, and the corresponding workload is reduced. The mark can represent the serial number of the tooth in the form of a numerical value, and / or represent the area of ​​concern as a selection area, such as the edge of the tooth or the edge of the pulp. For example, for conventional dentistry, a general recording method can be used to identify tooth positions. When marking teeth, a specific tooth can be determined by saving a mapping of coordinates and tooth positions. For example, {"index":23,offset:[751,850]} can identify the point 751 pixels to the right and 850 points below the upper left corner of the image, which corresponds to tooth No. 23. Similarly, multiple teeth can be marked in the image. The teeth marked here can be teeth to be diagnosed, or teeth with obvious features, such as the upper left teeth, upper right teeth, lower left teeth, and lower right teeth. In some embodiments of the present invention, the upper left teeth, upper right teeth, lower left teeth, and lower right teeth are provided, and the approximate distribution range of the teeth is determined by this, which reduces interference in the graphic analysis process and reduces the workload of image processing. In order to reduce the subsequent workload, the marked position is as close to the dentin area as possible.

[0061] The marking acquisition unit is used to obtain marking information including the focus point information and the reference dental arch information. The focus point information is specifically the marking of the tooth position and tooth category of the focus area, wherein the tooth category can refer to the common tooth position map, the tooth position is the position of the corresponding category of teeth on the image, and the dental arch information is obtained based on the focus point information and the oral image. In the case of a large number of samples, deep learning can be performed based on the information manually annotated by the oral image, such as using RNN or a neural network with an attention mechanism for learning, wherein the image annotated area is the dental arch shape annotated by a physician with professional experience. When constructing the corresponding recognition neural network, a physician with 1 to 2 years of professional experience annotates the image to obtain the annotated graphics, and then crops the image. The cropping size can be determined according to the requirements, such as selecting 15*15 pixels, and then a convolutional neural network is used to identify the cropped image to determine whether the corresponding image contains a manually marked image area, and the loss function is defined as the mean square error of the recognition error rate, and the approximate dental arch distribution in the image is determined in this way; it should be noted that the dental arch distribution here is a rough range value, but it can already meet the display of oral clinic medical records. The region of interest may be obtained by selecting the dental image based on interactive operations by the physician, or by identifying and annotating the image based on a model.

[0062] A spacing information unit is used to determine the value of the tooth spacing adjustment based on the oral image and tooth position information, wherein after the dental arch is determined, the corresponding adjustment of the spacing between the teeth is performed within the curve constrained by the dental arch, and the spacing can be obtained by identifying the tooth edges and calculating the distance between the teeth. The spacing between the teeth can be replaced by the center point image of the identified tooth contour, or based on the distance between the two tooth edges. Accordingly, the spacing of the corresponding teeth is adjusted in a preset model to make the displayed information consistent with the information reflected in the image.

[0063] The information drawing unit is used to determine the tooth area to be rendered in the canvas according to the marking information; the canvas of the present invention is the Canvas object of the browser, and the electronic medical record of the present invention is rendered in the browser, and the Canvas is used for drawing. When drawing, the preset model of the tooth is first rendered, and then the tooth model corresponding to the tooth position information in the tooth area to be rendered is adjusted based on the reference dental arch information, and the patch and texture are drawn, wherein the patch and texture drawing does not involve the correspondence with the image acquisition information, and the preset patch and texture are used for drawing. It should be noted that the patch and texture here do not involve real information, but are used as modifications of the application. For example, in the default case, no modification can be set, such as setting the selected tooth configuration to have a pink image as the filling background, and configuring discontinuous dotted oblique lines as the texture to remind relevant information. The relevant patch and texture can be defined by the user, such as the default patch is a milky white block, and the user can also set not to use texture and patch as needed.

[0064] The process of drawing the tooth area to be rendered includes:

[0065] Drawing a preset tooth model, wherein the tooth model includes a plurality of image regions composed of a plurality of points, and the plurality of image regions form a spatial arrangement similar to the tooth model;

[0066] Afterwards, the tooth model is adjusted according to the dental arch information so that the tooth model and the dental arch information have similar distributions, that is, the top of the tooth model is aligned with the tooth top information determined in the dental arch information;

[0067] The obtained tooth spacing is then used to adjust the spacing of each tooth model so that the spacing between the teeth corresponding to each tooth model corresponds to the spacing presented on the screen.

[0068] You can also set monitoring events here. For example, when the model receives a mouse event, the patch can change from the default color to yellow to highlight the object of the current interaction. You can also use obvious colors such as red to remind the interaction parties to pay attention to specific information.

[0069] In one embodiment of the present invention, the information drawing unit displays information in the following manner:

[0070] Draw on an element in the browser, the drawing area has a size of 1200*600 pixels;

[0071] The default model is loaded, which has the following definition: [

[0073] {

[0074] "name": "T0",

[0075] "index": [ 0

[0077] ],

[0078] "position": [

[0079] 100, 60

[0081] ],

[0082] "rotation": -15,

[0083] "source": {

[0084] "url": "Actual image address",

[0085] "size": 1

[0086] }

[0087] } ]

[0089] Among them, the basic information of the teeth is saved in an array, name is the name of the tooth, index is the serial number of the tooth, position is the offset of the tooth relative to the origin in the image, rotation is the rotation angle data of the tooth, which is specifically related to the dental arch, and source is used to indicate where the resource should be loaded from when requesting to render the tooth.

[0090] The default model contains 32 teeth. The corresponding teeth rendering image can be obtained through http request. The 32 teeth are drawn in the drawing area according to the default model of teeth.

[0091] Afterwards, the spacing information of the teeth is adjusted asynchronously, wherein the spacing information is obtained based on, for example, spacing information units, which is an array. The values ​​in the array represent the spacing between the maxillary teeth and the mandibular teeth. If the spacing is negative, it means that there are staggered traces between the teeth on the image; if the spacing is positive, it means that there are gaps between the teeth; accordingly, the image corresponding to the teeth is adjusted in the width direction or height direction of the image. When making adjustments, the images of all teeth in the maxilla or mandible should be adjusted. For example, when tooth No. 2 is moved 15 pixels to the left and downward according to the general recording method, teeth No. 2-16 are all moved 15 pixels to the left and downward. If the tooth direction is involved, individual adjustments are made to the teeth.

[0092] According to an embodiment of the present invention, the process of acquiring the marking information includes:

[0093] Obtaining feature points associated with the oral image based on tooth position information;

[0094] Determine the peripheral image area in the oral image based on the tooth position information, and determine the inflammation reference information associated with the tooth root based on the grayscale comparison method;

[0095] The reference dental arch information is determined according to the dental pulp distribution characteristics in the oral image.

[0096] In this way, reference marking information can be obtained from the image.

[0097] As mentioned above, when acquiring the full-mouth tooth position surface layer image, specific teeth in the image can be marked to reduce the cost of analysis. Although there are many technologies in the prior art that can be used for tooth position recognition and marking, the image data selected during the corresponding model training are inconsistent, and the image acquisition parameters also have an impact on the analysis results. Therefore, when performing analysis, the tooth position information is marked on the oral image. The tooth position information here can be incisors or molars. After the marking is completed, the area of ​​the processed image is basically determined. For example, when marking the incisors, the two sides of the incisors are canines, molars, etc.; when marking the end-side molars and incisors, the scope of image processing can be narrowed according to the determined tooth positions.

[0098] After obtaining the tooth position information, the tooth position and dental arch information of other teeth can be obtained based on the pre-trained network. As mentioned above, the approximate tooth image distribution area can be obtained according to the tooth position information, such as determining the distribution pixel range of the teeth based on the molars and incisors, and taking the horizontal coordinates of the two molars in the image as the coordinate range in the horizontal direction of the image, and taking the coordinate interval of the molars and incisors in the longitudinal direction as the coordinate range in the longitudinal direction of the image; if the incisor is used for calculation, the incisor can be used as the far point, and the pixel area of ​​the specified length can be intercepted to the top and left and right sides of the image for calculation, and so on, to achieve the interception of tooth-related areas.

[0099] After the tooth-related area is determined, the corresponding surrounding image area is also determined, and the inflammation reference information associated with the tooth root is determined based on the grayscale comparison method. Because the grayscale value of the area corresponding to oral inflammation on the image is greater than that of the non-inflammation area, the corresponding area is marked to obtain information such as apical periodontitis to be identified. It should be noted that some inflammation cannot be obtained through the full-mouth tooth surface layer image used in the present invention, and other methods should be combined to obtain the corresponding information. The relevant information is only used for reminders and cannot be understood as a purpose other than reference information.

[0100] Furthermore, based on the tooth position information and the associated feature points, the distribution of each tooth can be obtained. In the full-mouth tooth position curved layer image, the grayscale value of the pulp is lower than that of the tooth body and is correlated with the position of the tooth body. Based on this, the present invention uses the distribution characteristics of the pulp to determine the distribution characteristics of the dental arch. The traditional determination of the shape of the dental arch relies on the knowledge of professionals, and when there is a lack of corresponding reference objects, it is impossible to quickly obtain a reference marker. Based on the neural network recognition of the dental arch mentioned above, the present invention uses the characteristics of the pulp to be more obvious in the image, and uses it as a feature to identify the dental arch. Correspondingly, when training the neural network, the characteristics of the pulp are used as input. For example, the pulp image corresponding to each tooth in the upper jaw uses the coordinates of the center point of the pulp as a feature point to construct an input vector, and the output vector is the coordinate point of the dental arch. Then, a neural network based on the pulp feature recognition of the dental arch can be obtained, and the reference dental arch information can be obtained.

[0101] Please refer to Figure 2 , according to one embodiment of the present invention, the associated characteristic points include the top position of the tooth, the root position and the size characteristics of the pulp tissue;

[0102] The feature points associated with the oral image obtained according to the tooth position information include:

[0103] Determine a processing base point of the oral image according to the marked tooth position information, wherein the processing base point includes an image area corresponding to at least one tooth;

[0104] Determining tooth edges in an oral image based on a grayscale histogram;

[0105] Determining the position of the top of the tooth and the position of the root of the tooth based on the shape defined by the edge of the tooth;

[0106] The size characteristics of the dental pulp tissue corresponding to the tooth are determined based on the shape defined by the tooth edge.

[0107] As a solution, the present invention obtains associated feature points including common positions of teeth. When rendering, the top position and root position of the tooth can be used to control the scaling of the model, and the size characteristics of the pulp tissue are associated with the image area of ​​the pulp tissue to calculate the pulp distribution characteristics. When calculating the spacing, the pixel points corresponding to the size characteristics of the pulp tissue are used to calculate the center point of the image and adjust the spacing of the teeth.

[0108] Among them, the tooth area to be processed is determined according to the tooth position information. Since the grayscale value of the tooth's dentin in the panoramic image is higher than other areas, it appears as a brighter part (the grayscale value is generally greater than 150), while the grayscale value of the pulp color is higher (the grayscale value is generally lower than 130), which appears to be darker than the surrounding dentin and has the characteristics of continuous distribution; the color distribution of the gums, the rest of the human body's bones and muscles other than the teeth in the image is uneven, such as the grayscale value of the pixels of the rest of the oral cavity other than the teeth in the panoramic film is generally lower than 75. In order to avoid misunderstandings introduced by the image acquisition link, the present invention mainly crops the image area based on this, obtains several sub-images, calculates the grayscale histograms of the sub-images respectively, compares the grayscale histogram of the currently selected tooth position instead of the aforementioned grayscale value, obtains the similarity between the grayscale histogram of the sub-image and the tooth position, and determines whether the corresponding image area is the edge of the tooth based on the similarity. In this way, the approximate position of the edge can be determined more quickly.

[0109] When performing similarity comparison, the cosine algorithm can be used. This method can reduce the processing workload. When performing image cropping or segmentation, resolutions such as 10*10, 15*15, 20*20, and 30*30 pixels can be selected. In this way, some areas are obtained, and the grayscale histogram approximation of the processing base point is greater than the threshold. In most cases, the line connecting the center points of the corresponding areas is the tooth edge. On this basis, the internal shape of the tooth edge constraint is the tooth image. Based on the marked tooth position, the top or bottom of the tooth image can be determined as the tooth top / root. Since it is difficult to render size and defect information on the tooth position map, defect features are not processed, and only position information is retained. Other information is obtained by providing an entry. In addition, some areas may be misidentified, such as the image between the pulp and the dentin, which may be misidentified, but since the error caused by these points when used to calculate the position of the image center point is small, no further processing is required.

[0110] Approximately, since the grayscale histogram of the pulp tissue is more obvious than other locations, the edge of the pulp tissue inside the tooth image can be obtained by an approximate method, that is, a preset pulp boundary image can be selected as a reference, and a grayscale method map can be calculated to segment the internal tooth image, and the grayscale histogram of the segmented image and the grayscale histogram calculated from the preset image are calculated to obtain the boundary of the pulp tissue. Since this process is carried out inside the tooth image, the corresponding error possibility is reduced. In the case where the boundary of the pulp tissue cannot be obtained due to dose reasons, since some features of the data source for the learning of the corresponding neural network used to identify the dental arch do not completely rely on the pulp features, the information of the dental arch can still be obtained.

[0111] According to one embodiment of the present invention, determining the tooth edge in the oral image based on the grayscale histogram includes:

[0112] According to the processing base point in the oral image, a reference grayscale histogram at the processing base point is obtained;

[0113] According to a reference grayscale histogram and an oral image within a predetermined area, an edge area corresponding to the oral image is determined, wherein the predetermined area is determined based on a processing base point and an extension direction of teeth in the oral image, and in the oral image, the lower jaw to the upper jaw is a first direction, and the upper jaw to the lower jaw is a second direction, and the extension direction includes an arrangement direction from the processing base point to the molar teeth.

[0114] In this way, the tooth edges in the oral cavity image can be obtained.

[0115] A specific example is used for explanation. The marked tooth position information is the incisor, the grayscale histogram at the processing base point is recorded as the first grayscale histogram, the coordinates of the processing base point are marked as (a, b), and the incisor to a molar (a-1000, b-300) is taken as the origin (1000 pixels to the left of the processing base point and with an upward offset of 300 pixels in the second direction), a 2000*300 pixel area is obtained, and the image is divided into 10*10 sub-images, and the cosine approximations of the sub-images and the first grayscale histogram are compared in turn to obtain sub-images with cosine approximations greater than 0.85. The sub-images are grouped according to whether the corresponding coordinates of the sub-images are adjacent in the figure (here they can also be clustered according to the number of teeth involved in the processing) to obtain several groups of sub-images, and the center points of the sub-images in the group are connected to obtain the edges of each tooth.

[0116] Other edge recognition models can also be used for recognition here, such as those based on YOLO and U-Net.

[0117] According to an embodiment of the present invention, the step of determining the edge area corresponding to the oral cavity image includes:

[0118] Segmenting the oral cavity image within a predetermined area to obtain a plurality of segmented sub-images, wherein the sub-images have a pixel size of not less than 10*10;

[0119] Obtaining grayscale histograms of several segmented sub-images respectively, determining edge sub-images for edge region division based on the approximation between the grayscale histogram of each sub-image and the reference grayscale histogram, and determining a closed curve based on the image center of the edge sub-image to obtain a tooth region;

[0120] The margins of individual teeth are obtained according to the tooth regions.

[0121] In this way, the area passing through the tooth image can be obtained. However, in the case of irregular teeth, the upper edges of the image may overlap. In this case, clustering or LOF can be used to distinguish them. In this way, points are divided and points in the same area are connected to obtain the tooth area.

[0122] At this point, other pre-trained tooth segmentation neural networks can be further used to perform edge recognition on the tooth area to obtain clearer boundaries. The corresponding boundaries are the edges of the teeth. The determination of the edges can meet the requirements for determining the pulp features and can be used to calculate the intervals between teeth.

[0123] A preset value can be set to determine which images can be used for edge area division, such as setting a threshold value of 0.8, and using images with an approximation greater than 0.8 as edge sub-images. Then, a closed curve is determined based on the image center of the edge sub-image to obtain the tooth area. The closed curve drawing process here does not need to consider the specific area or order, such as selecting the center of any edge sub-image as the starting point or calculation basis, and using other edge sub-images as iteration objects. The center point of the nearest edge sub-image is obtained from the iteration object multiple times as the next connection point of the curve, and it is removed from the iteration object until all edge sub-images in the iteration object are processed, and the center of the initially selected edge sub-image is used as the end point of the closed curve to form a closed curve or image. In this way, the edges of each tooth are obtained from the tooth area determined by the edge sub-image, which can be used to roughly determine the position of the tooth.

[0124] According to an embodiment of the present invention, the approximation is obtained by a cosine approximation algorithm.

[0125] The cosine approximation algorithm can be used to obtain the approximation of the grayscale histograms between two images and to determine whether the grayscale distribution of an image is similar to the tooth edge.

[0126] According to one embodiment of the present invention, the processing base point is the central incisor of the upper or lower jaw.

[0127] In this way, the base point can be determined more quickly, and the tooth image area can be acquired based on the base point, especially by expanding in the direction from incisor to molar, and acquiring images of the positions of other teeth based on the first direction or the second direction, so that the tooth area showing an arc-shaped distribution is selected.

[0128] According to an embodiment of the present invention, determining the reference dental arch information according to the dental pulp distribution characteristics in the oral CT image includes:

[0129] obtaining an image of the interior of the tooth edge based on the tooth edge;

[0130] Determine the pulp distribution area inside each tooth based on the pulp grayscale threshold;

[0131] Determine the distribution characteristics of the dental pulp inside each tooth according to the coordinates of the center point of the dental pulp distribution area inside each tooth;

[0132] The characteristics of the dental arch are determined based on the distribution characteristics of the dental pulp.

[0133] In this way, the image area of ​​the dental pulp can be approximately obtained, and the dental arch features can be determined based on the image area of ​​the dental pulp.

[0134] The grayscale value of the dental pulp is lower than that of the tooth body, and appears darker than the tooth body on the image. The boundary of the dental pulp inside the tooth body is determined based on a method similar to that of determining the edge of the tooth body.

[0135] In one embodiment, the image of the tooth is segmented into sub-images of 10*10 pixels, and the cosine approximations of the sub-images and the first grayscale histogram of the preset pulp boundary are compared in turn to obtain sub-images with a cosine approximation greater than 0.85, and the center points of the sub-images in the group are connected to obtain the edge of the pulp image.

[0136] Furthermore, the center point of the image is calculated, that is, the mean of the x values ​​of all points in the image and the mean of the y values ​​of all points, to obtain the center point of the image, where x and y are calculated with the upper left corner of the image as the origin, x is the value in the length direction of the offset origin, and y is the value in the height direction of the offset origin.

[0137] After calculating the distribution characteristics of the pulp of all teeth, the characteristics of the dental arch are obtained through the network.

[0138] According to an embodiment of the present invention, the process of obtaining inflammation reference information includes:

[0139] Acquire tooth root images based on oral CT images;

[0140] Binarize the tooth root image;

[0141] The apical periodontitis detection network is used to identify the tooth root image, and the recognition result of the tooth root image is obtained as the inflammation reference information.

[0142] The neural network for identifying periapical inflammation images is obtained in the following way:

[0143] First, historical oral images collected in the hospital are screened by symptoms, combined with medical records or labeled based on images. The labelers are the attending physicians. When labeling, the main content is the root area with inflammation. Among them, the area without inflammation is not labeled, but used as a negative sample set for training during testing. A fixed image capture window is used during labeling, such as 100*100 pixels or 125*125 pixels, which can be selected according to needs.

[0144] Afterwards, positive and negative samples are extracted from the original oral images based on the results of the annotations. The positive samples are based on the annotated areas. Since they are obtained from in-hospital instruments, the image size and source are controllable. After binarizing the corresponding images, a batch of images that are manually marked as having inflammation are obtained. On this basis, since the positive sample images contain the characteristics of the tooth root, all the positive sample images are used as training sets to construct an RNN neural network, and the areas captured in the original images are used for identification. In this way, information related to apical periodontitis can be identified.

[0145] Please refer to Figure 3 According to one embodiment of the present invention, an interactive outpatient oral medical record display method includes:

[0146] Acquire an oral cavity image, wherein the oral cavity image is a full-mouth tooth position curved surface layer image and contains tooth position information of at least one tooth;

[0147] Acquire marking information associated with the oral image, wherein the marking information includes focus point information and reference dental arch information;

[0148] Obtain spacing information based on oral images and tooth position information;

[0149] Determine the tooth area to be rendered in the canvas according to the marking information, adjust the tooth model in the tooth area to be rendered and corresponding to the tooth position information based on the reference dental arch information, determine the spacing of the tooth model based on the spacing information, draw patches on the tooth model corresponding to the tooth position information, and draw the focus area patch in the canvas based on the rendering rules corresponding to the focus point information.

[0150] An embodiment of the present application further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed on an electronic device, the electronic device executes the aforementioned method.

[0151] An embodiment of the present application further provides a computer program product, including: a computer program code, when the computer program code is executed on an electronic device, the electronic device executes the aforementioned method.

[0152] An embodiment of the present application further provides a chip, comprising: a processor, configured to call and run a computer program from a memory, so that an electronic device equipped with the chip executes the aforementioned method.

[0153] Through the description of the above implementation mode, those skilled in the art can understand that for the convenience and simplicity of description, only the division of the above functional modules is used as an example. In practical applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above.

[0154] It should be understood that the devices and processes disclosed in the several embodiments provided in the present application can be implemented in other ways. The device embodiments described above are merely schematic. For example, the division of modules or units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another device. In addition, some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between each other shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0155] The units described as separate components may or may not be physically separated. The components shown as units may be one physical unit or multiple physical units. That is, they may be located in one place or distributed in multiple different places. Some or all of the units may be selected to achieve the purpose of this solution according to actual needs.

[0156] In addition, each functional unit in each embodiment of the present application may be integrated into a processing unit; or may exist physically separately; or some units may be integrated into one unit, and some units may exist physically separately. The above-mentioned integrated units may be implemented in the form of hardware or in the form of software functional units.

[0157] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a readable storage medium. Based on this understanding, all or part of the technical solution of the embodiment of the present application can be embodied in the form of a software product. The software product is stored in a storage medium. The software product includes several instructions to enable a device (which can be a single-chip microcomputer, chip, etc.) or a processor to perform all or part of the steps of the various embodiments of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a ROM, a RAM, a magnetic disk, or an optical disk.

[0158] It should be noted that all or part of the above-mentioned embodiments provided in the present application (for example, part or all of any feature) can be arbitrarily combined or used in combination with each other.

[0159] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. An interactive outpatient oral medical record display device, characterized in that: include: An image acquisition unit, used for acquiring an oral cavity image, wherein the oral cavity image is a full-mouth tooth position curved surface layer image and contains tooth position information of at least one tooth; A marking acquisition unit, used for acquiring marking information associated with the oral image, wherein the marking information includes focus point information and reference dental arch information; A spacing information unit, used to obtain spacing information according to the oral image and tooth position information; An information drawing unit, used for determining a tooth area to be rendered in the canvas according to the marking information, adjusting a tooth model in the tooth area to be rendered and corresponding to the tooth position information based on the reference dental arch information, determining a spacing of the tooth model based on the spacing information, drawing a patch on the tooth model corresponding to the tooth position information, and drawing a focus area patch in the canvas based on a rendering rule corresponding to the focus point information; The focus point information is the marking of the tooth position and tooth type in the focus area; The focus area is obtained by selecting the dental image based on interactive operations by the physician, or by identifying and marking the image based on a model.

2. The interactive outpatient oral medical record display device according to claim 1, characterized in that: The process of obtaining the marking information includes: Obtaining feature points associated with the oral image based on tooth position information; A tooth root image is obtained according to an oral CT image, the tooth root image is binarized, and the tooth root image is recognized using an apical inflammation detection network, and the recognition result of the tooth root image is obtained as inflammation reference information; Determine the reference dental arch information according to the dental pulp distribution characteristics in the oral image; The associated characteristic points include the top position of the tooth, the root position and the size characteristics of the pulp tissue; The feature points associated with the oral image obtained according to the tooth position information include: Determine a processing base point of the oral image according to the marked tooth position information, wherein the processing base point includes an image area corresponding to at least one tooth; Determining tooth edges in an oral image based on a grayscale histogram; Determining the position of the top of the tooth and the position of the root of the tooth based on the shape defined by the edge of the tooth; The size characteristics of the dental pulp tissue corresponding to the tooth are determined based on the shape defined by the tooth edge.

3. The interactive outpatient oral medical record display device according to claim 2, characterized in that: The method of determining the tooth edge in the oral image based on the grayscale histogram comprises: According to the processing base point in the oral image, a reference grayscale histogram at the processing base point is obtained; According to a reference grayscale histogram and an oral image within a predetermined area, an edge area corresponding to the oral image is determined, wherein the predetermined area is determined based on a processing base point and an extension direction of teeth in the oral image, and in the oral image, the lower jaw to the upper jaw is a first direction, and the upper jaw to the lower jaw is a second direction, and the extension direction includes an arrangement direction from the processing base point to the molar teeth.

4. The interactive outpatient oral medical record display device according to claim 3, characterized in that: The step of determining the edge area corresponding to the oral cavity image comprises: Segmenting the oral cavity image within a predetermined area to obtain a plurality of segmented sub-images, wherein the sub-images have a pixel size of not less than 10*10; Obtaining grayscale histograms of several segmented sub-images respectively, determining edge sub-images for edge region division based on the approximation between the grayscale histogram of each sub-image and the reference grayscale histogram, and determining a closed curve based on the image center of the edge sub-image to obtain a tooth region; The margins of individual teeth are obtained according to the tooth regions.

5. The interactive outpatient oral medical record display device according to claim 4, characterized in that: The approximation is obtained by a cosine approximation algorithm.

6. The interactive outpatient oral medical record display device according to claim 2, characterized in that: The processing base point is the central incisor of the upper jaw or the lower jaw.

7. The interactive outpatient oral medical record display device according to claim 2, characterized in that: Determining the reference dental arch information according to the dental pulp distribution characteristics in the oral image includes: obtaining an image of the interior of the tooth edge based on the tooth edge; Determine the pulp distribution area inside each tooth based on the pulp grayscale threshold; Determine the distribution characteristics of the dental pulp inside each tooth according to the coordinates of the center point of the dental pulp distribution area inside each tooth; The characteristics of the dental arch are determined based on the distribution characteristics of the dental pulp.

8. An interactive outpatient oral medical record display method, comprising: Acquire an oral cavity image, wherein the oral cavity image is a full-mouth tooth position curved surface layer image and contains tooth position information of at least one tooth; Acquire marking information associated with the oral image, wherein the marking information includes focus point information and reference dental arch information; Obtain spacing information based on oral images and tooth position information; Determine the tooth area to be rendered in the canvas according to the marking information, adjust the tooth model in the tooth area to be rendered and corresponding to the tooth position information based on the reference dental arch information, determine the spacing of the tooth model based on the spacing information, draw a patch on the tooth model corresponding to the tooth position information, and draw the focus area patch in the canvas based on the rendering rule corresponding to the focus point information; The focus point information is the marking of the tooth position and tooth type in the focus area; The focus area is obtained by selecting the dental image based on interactive operations by the physician, or by identifying and marking the image based on a model.

Images