Oral panoramic image processing method and device, electronic equipment and readable storage medium

Abstract

This disclosure provides a method, apparatus, electronic device, and readable storage medium for processing panoramic oral images, relating to the field of computer technology, specifically to the fields of dental orthodontics and computer-aided design. The specific implementation involves: outputting a panoramic oral image; responding to user operations based on the panoramic oral image to obtain information about a specified region on the panoramic oral image; and, based on the panoramic oral image and the specified region information, using a neural network model to output disease information from the panoramic oral image.

Description

Technical Field

[0001] This disclosure relates to the field of computer technology, specifically to the fields of dental orthodontics and computer-aided design. Background Technology

[0002] With the continuous development of computer science, the dental field is increasingly leveraging computer technology. For example, in orthodontics, panoramic images of the oral cavity are required. These images can clearly and completely display the entire maxilla and mandible, the condition of the maxillary and mandibular dentition, the alveolar bone, the maxillary sinus cavity, sinus walls, sinus floor, and temporomandibular joint, providing accurate and effective assistance in the diagnosis of diseases surrounding the jawbone. Using deep learning neural network models, diseases in these panoramic oral images can be automatically identified.

[0003] However, because the training method of neural network models determines that they cannot guarantee that the recognition results will always be correct, in most cases doctors may need to further review the output results of the neural network models. This process is time-consuming and prone to errors, thus reducing the efficiency and reliability of oral disease recognition. Summary of the Invention

[0004] This disclosure provides a method, apparatus, electronic device, and readable storage medium for processing panoramic oral images.

[0005] According to one aspect of this disclosure, a method for processing panoramic oral images is provided, comprising: Output a panoramic image of the oral cavity; In response to a user operation based on the panoramic oral image, information about a specified region on the panoramic oral image is obtained; Based on the panoramic oral image and the information of the specified region, a neural network model is used to output disease information on the panoramic oral image.

[0006] According to another aspect of this disclosure, a processing apparatus for panoramic oral images is provided, comprising: Image output unit, used to output panoramic images of the oral cavity; Information prompting unit is used to obtain information about a specified area on the oral panoramic image in response to user operation based on the oral panoramic image; The disease output unit is used to output disease information on the oral panoramic image based on the oral panoramic image and the information of the specified region, using a neural network model.

[0007] According to another aspect of this disclosure, an electronic device is provided, comprising: At least one processor; and A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the methods described above and any possible implementations.

[0008] According to another aspect of this disclosure, a non-transitory computer-readable storage medium is provided storing computer instructions for causing the computer to perform the methods described above and any possible implementation thereof.

[0009] According to another aspect of this disclosure, a computer program product is provided, comprising a computer program that, when executed by a processor, implements the aspects and any possible implementations described above.

[0010] As can be seen from the above technical solution, the embodiments of this disclosure output a panoramic image of the oral cavity, and then, in response to user operations based on the panoramic image, obtain information about a specified region on the panoramic image. This enables the use of a neural network model to output disease information on the panoramic image based on the panoramic image and the specified region information. Because the specified region information on the panoramic image is additionally provided based on user operations, the output results of the neural network model are more accurate, and the operation is simple and convenient, thereby improving the efficiency and reliability of intraoral disease identification.

[0011] In addition, by using the technical solution provided in this disclosure, doctors can provide relevant information about the problem area on the panoramic image of the oral cavity through simple interface interaction, which can effectively provide prompt information to the neural network model, thereby improving the reliability of the neural network model in recognizing oral diseases.

[0012] In addition, by using the technical solution provided in this disclosure, doctors can adjust the disease information on the panoramic oral image output by the neural network model, thereby enabling convenient and automated intervention of the neural network model. A large amount of doctor intervention data can be effectively applied to subsequent training of a more accurate neural network model.

[0013] In addition, the technical solutions provided in this disclosure can effectively improve the user experience.

[0014] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of this disclosure, nor is it intended to limit the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description

[0015] To more clearly illustrate the technical solutions in the embodiments of this disclosure, the accompanying drawings used in the description of the embodiments or prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this disclosure. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort. The drawings are used to better understand this solution and do not constitute a limitation of this disclosure. Wherein: Figure 1 This is a schematic diagram based on the first embodiment of the present disclosure; Figure 2 This is a schematic diagram according to the second embodiment of the present disclosure; Figure 3 This is a block diagram of an electronic device used to implement the oral panoramic image processing method of the embodiments of this disclosure. Detailed Implementation

[0016] The exemplary embodiments of this disclosure are described below with reference to the accompanying drawings, including various details of the embodiments to aid understanding, and should be considered merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of this disclosure. Similarly, for clarity and brevity, descriptions of well-known functions and structures are omitted in the following description.

[0017] Obviously, the described embodiments are only some, not all, of the embodiments disclosed herein. All other embodiments obtained by those skilled in the art based on the embodiments of this disclosure without inventive effort are within the scope of protection of this disclosure.

[0018] It should be noted that the terminals involved in the embodiments of this disclosure may include, but are not limited to, smart devices such as computers (PCs), mobile phones, personal digital assistants (PDAs), wireless handheld devices, and tablet computers; display devices may include, but are not limited to, personal computers, televisions, and other devices with display functions.

[0019] Furthermore, the term "and / or" in this article is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. Additionally, the character " / " in this article generally indicates that the preceding and following related objects have an "or" relationship.

[0020] In the field of oral medicine, panoramic oral imaging serves as an important diagnostic tool, widely used in the diagnosis and treatment planning of dental diseases. Panoramic images provide complete visual information of the patient's oral cavity, aiding dentists in conducting comprehensive examinations. However, because panoramic oral images typically contain a wealth of structural information, including teeth, bones, and other soft tissues, quickly and accurately identifying lesion areas and making effective disease diagnoses remains a technical challenge.

[0021] Traditional panoramic oral imaging analysis relies primarily on manual diagnosis by physicians. Physicians observe panoramic images and combine this with clinical experience to identify abnormal areas in the oral cavity, such as dental caries, periodontitis, and bone resorption. This method not only depends on the physician's professional knowledge and experience but is also affected by their skill level and fatigue, making it prone to missed or misdiagnosed cases. With the increasing complexity of oral disease types and cases, the efficiency and accuracy of manual diagnostic methods have faced significant challenges.

[0022] Currently, with the development of artificial intelligence and deep learning technologies, automated image analysis methods based on neural networks have been gradually introduced into the field of oral image processing. In particular, by training neural network models and combining them with the characteristics of medical images, automatic identification of disease areas can be achieved. However, existing image analysis methods still have certain limitations in terms of accuracy, robustness, and processing speed. Especially when processing complex panoramic images of the oral cavity, the algorithms have a weak ability to identify some blurred or small lesion areas, which may lead to inaccurate or misjudgment of diagnostic results. Correction of the diagnostic results still requires the assistance of doctors, thus reducing the efficiency and reliability of oral disease identification.

[0023] Therefore, there is an urgent need to provide an effective method to improve the efficiency and reliability of oral disease identification.

[0024] Figure 1 This is a schematic diagram based on the first embodiment of the present disclosure, as shown below. Figure 1 As shown.

[0025] 101. Output a panoramic image of the oral cavity.

[0026] 102. In response to a user operation based on the panoramic oral image, obtain information about a specified region on the panoramic oral image.

[0027] 103. Based on the panoramic oral image and the information of the specified region, a neural network model is used to output disease information on the panoramic oral image.

[0028] The disease information on the panoramic oral image may include, but is not limited to, the following: The disease area on the panoramic image of the oral cavity, for example, the geometric features of the disease area such as coordinates, size, and shape; or The disease area on the panoramic image of the oral cavity and the corresponding disease description, such as the disease name (e.g., dental caries, alveolar bone resorption), the degree of the disease (e.g., mild, severe), and the formation of the disease.

[0029] Thus, by using the information of a specified region on the oral panoramic image obtained in response to user operations based on the output oral panoramic image as supplementary prompt information for the oral panoramic image, and inputting it together with the oral panoramic image into a pre-trained neural network model, the neural network model is able to output accurate disease information on the oral panoramic image.

[0030] It should be noted that some or all of the execution entities of 101 to 103 can be applications located on the local terminal, or they can be functional units such as plug-ins or software development kits (SDKs) set in applications located on the local terminal, or they can be processing engines located on the network-side server, or they can be distributed systems located on the network side, such as processing engines or distributed systems in network-side oral panoramic image processing devices, etc. This embodiment does not impose any special limitations on this.

[0031] It is understood that the application can be a native program installed on the local terminal, or it can be a web application of a browser on the local terminal. This embodiment does not limit this.

[0032] In this way, by outputting a panoramic image of the oral cavity, and then responding to user operations based on the panoramic image, information about a specified region on the panoramic image is obtained. This enables the use of a neural network model to output disease information on the panoramic image based on the panoramic image and the specified region information. Because the specified region information on the panoramic image is provided additionally based on user operations, the output results of the neural network model are more accurate, and the operation is simple and convenient, thereby improving the efficiency of disease identification in the oral cavity.

[0033] Optionally, in one possible implementation of this embodiment, before step 101, an image acquisition device, such as a panoramic X-ray machine or other oral imaging device, can be used to acquire panoramic images of the oral cavity and obtain complete image data of the patient's oral cavity.

[0034] It should be noted that this disclosure does not limit the specific method of outputting panoramic images of the oral cavity, and the appropriate method can be selected according to the actual situation.

[0035] Optionally, in one possible implementation of this embodiment, in step 101, a panoramic image of the oral cavity inside the cavity can be displayed on a screen.

[0036] At this point, based on the output panoramic oral image, the user (i.e., the doctor) can select specific areas of interest or areas deemed abnormal that require further attention or diagnosis. For example, the doctor can input information using input devices such as a mouse or touchscreen to provide relevant information, which can serve as information about a specific area on the panoramic oral image. This information could include at least one of the following: the location of the specified area on the panoramic oral image and a prompt related to that area, for subsequent processing.

[0037] In a specific implementation, users can view the output panoramic oral image and, through clicking the mouse or long-pressing and dragging the cursor, obtain image indicators on the panoramic oral image indicating the location of the disease they believe to be present. For example, the image indicators can be at least one of regular geometric shapes, irregular shapes, points, and lines to indicate the location of the disease that the user believes to be present.

[0038] In another specific implementation process, based on the previous specific implementation process, users can further provide a text description of the location they believe to have the disease through voice or keyboard input, for example, at least one of the following text descriptions—the location of the disease, the name of the disease, and the severity of the disease—to prompt the user to the location of the disease they believe to have.

[0039] It should be noted that this disclosure does not limit the specific method for obtaining information about a specified region on the panoramic oral image in response to user operations based on the panoramic oral image, and the appropriate method can be selected according to the actual situation.

[0040] Optionally, in one possible implementation of this embodiment, in step 102, the location information of a designated area on the panoramic oral image can be determined in response to a user operation based on the panoramic oral image, and further, prompt content for the designated area can be obtained.

[0041] At this time, the location information of the specified area on the obtained panoramic image of the oral cavity, or the location information of the specified area on the panoramic image of the oral cavity and the prompt content of the specified area, can be used as the descriptive information of the specified area to indicate the relevant content of the specified area.

[0042] Optionally, in one possible implementation of this embodiment, the neural network model used in step 103 can be trained and constructed using a Segment Anything Mode (SAM) structure, or it can be trained and constructed using any segmentation network structure capable of inputting prompt information. This embodiment does not impose any particular limitation on this.

[0043] Specifically, the neural network model used can consist of a first encoder, a second encoder, and a decoder.

[0044] The first encoder can be a prompt encoder, used to process the input prompt information, i.e., information about a specified region on the panoramic oral image. The first encoder converts the information about the specified region on the input panoramic oral image into a prompt embedding as output.

[0045] The second encoder can be an image encoder, used to process the input image, namely the panoramic image of the oral cavity. The second encoder converts the input panoramic image of the oral cavity into an image embedding output.

[0046] The decoder, which can be a mask decoder, processes the embedding codes output by the first and second encoders. The decoder decodes the cue embedding codes output by the first encoder and the image embedding codes output by the second encoder to obtain multiple recognition results and a score for each result. The score represents the reliability of the corresponding recognition result, and the recognition result with the highest score exceeding a threshold is output as the final result of the neural network model.

[0047] The recognition result may include a binary image, i.e., a mask, used to mark disease areas on the panoramic oral image, or, in addition to including a binary image, i.e., a mask, used to mark disease areas on the panoramic oral image, it may further include descriptive content describing the disease area. The binary image can serve as a unique identifier for the disease area.

[0048] Furthermore, different contents in the recognition results can be associated in various ways. For example, they can be associated using a one-hot encoded one-dimensional array.

[0049] By designing the neural network model structure, it is possible to load a pre-trained model that has been pre-trained on a large-scale dataset, thus inheriting some of the segmentation capabilities of the pre-trained model.

[0050] Optionally, in one possible implementation of this embodiment, before step 103, a step of constructing a neural network model using a pre-trained segmentation network model may be further included.

[0051] It should be noted that this disclosure does not limit the specific method of constructing a neural network model using a pre-trained segmentation network model, and the method can be selected according to the actual situation.

[0052] During the construction phase, pre-trained segmentation network models, such as the SAM model, can be used, employing panoramic oral images labeled with disease regions as training data. Furthermore, the training data can also be further annotated with the disease descriptions corresponding to those disease regions.

[0053] By using preprocessing functions, the training data (i.e., panoramic images of the oral cavity, binary images of disease areas, and disease descriptions) are diversified to increase the diversity of the data, thereby improving the generalization ability of the model.

[0054] For example, preprocessing functions used to add random noise, such as adding a small amount of random offset to point coordinates or box positions.

[0055] Alternatively, for example, preprocessing functions used to adjust for scale changes, such as randomly adjusting the box size or adding subtle errors in annotations by different doctors.

[0056] Alternatively, for example, preprocessing functions for rotation and mirroring, such as rotating, flipping horizontally, and flipping vertically simultaneously on an image and its corresponding points and boxes.

[0057] The pre-trained segmentation network model processes the input training data and outputs the training results. The training results are compared with the actual labeled binary images and disease descriptions. Based on the comparison results, the weights of the segmentation network model are updated using the stochastic gradient descent algorithm to obtain the neural network model.

[0058] To improve the performance of the neural network model, in a specific implementation, an optimization strategy can be introduced, namely, freezing the weights of the encoder and the first part of the decoder in the pre-trained segmentation network model. In this way, the weights of the second part of the decoder in the segmentation network model can be updated using the stochastic gradient descent algorithm to obtain the neural network model.

[0059] Since the parameters of the encoder and its first decoder parts of the segmentation network model remain unchanged during training, it retains its learned feature extraction capabilities. This method effectively reduces overfitting during training and stabilizes the network model's performance.

[0060] Meanwhile, the stochastic gradient descent algorithm iterates continuously, calculating the error and adjusting the weights of the second-part decoder based on that error, until the output of the segmentation network model reaches the expected accuracy. In this way, the second-part decoder can be trained more accurately on the current dataset of panoramic oral images, improving its ability to identify disease areas within the oral cavity.

[0061] During the update of the second part of the decoder, the segmentation network model combines the input panoramic oral image with information about designated regions on the panoramic image to learn how to more accurately extract disease features from the panoramic oral image and identify the size, shape, and other pathological features of the lesion area. This optimization process further improves the overall recognition performance of the obtained neural network model, especially when dealing with complex oral diseases, enabling it to better handle different panoramic oral images and disease variations.

[0062] The trained neural network model can provide more accurate disease regions and descriptions when outputting disease information, effectively helping doctors to determine the location of lesions faster and more accurately, and provide treatment suggestions. By freezing some weights of the neural network model and optimizing some decoder weights during training, not only can the accuracy of intraoral disease recognition be improved, but the generalization ability of the neural network model can also be further enhanced, enabling the neural network model to maintain high recognition accuracy and reliability in panoramic oral images of various patients.

[0063] It should be noted that this disclosure does not limit the specific method of outputting disease information on the panoramic oral image based on the panoramic oral image and the information of the specified region using a neural network model, and the method can be selected according to the actual situation.

[0064] Optionally, in one possible implementation of this embodiment, in step 103, disease information on the oral panoramic image can be obtained using a neural network model based on the oral panoramic image and the specified region information, and then the disease information on the oral panoramic image can be visualized and output.

[0065] By analyzing panoramic images of the oral cavity using a deep learning neural network model and combining it with information from specified regions, the system automatically identifies lesion areas within these images. The neural network model accurately extracts disease areas through image segmentation techniques and can further output corresponding disease information based on factors such as the type, morphology, and severity of the disease area, including conditions like dental caries, periodontitis, and alveolar bone resorption. The obtained disease information includes the geometric features of the lesion area, such as its coordinates, size, and shape. This information, combined with a medical knowledge base, generates a disease description of the lesion area, helping doctors understand the specific condition of the lesion.

[0066] The disease information on the output panoramic oral image can be displayed on the user interface using visualization technology to ensure that doctors can quickly identify the accurate location and extent of the lesion, as well as the disease-related description.

[0067] In a specific implementation, the diseased area can be displayed on the panoramic oral image using a colored semi-transparent mask to highlight the diseased area. The masked area is the diseased area. Alternatively, the non-diseased area can also be displayed on the panoramic oral image using a colored semi-transparent or opaque mask to cover the non-diseased area. The uncovered area is the diseased area. Or, the edge of the diseased area can be displayed on the panoramic oral image using a colored semi-transparent irregular line to highlight the edge of the diseased area. The area enclosed by the edge is the diseased area. Other visualization techniques can also be used to display the diseased area on the panoramic oral image. This embodiment does not impose any particular limitations on this.

[0068] In another specific implementation, the disease description corresponding to the disease area can be displayed on the panoramic oral image in a position that does not obscure the disease area, or it can be displayed in association with a separate text file, or other visualization techniques can be used to display the disease description corresponding to the disease area. This embodiment does not impose any particular limitations on this.

[0069] Furthermore, this implementation can also support interactive functions, allowing doctors to fine-tune or confirm the disease information based on the visualized output. After visually outputting the disease information on the panoramic oral image, it can further respond to user operations based on the disease information on the panoramic oral image, adjusting the disease information on the panoramic oral image.

[0070] In the image, doctors can adjust the location, size, or description of the disease area as needed.

[0071] For example, doctors can directly adjust the location and size of the disease area by manipulating its geometric features through the interface, such as the position or boundaries. They can also indirectly adjust these features by modifying the model segmentation threshold of the neural network model. A higher segmentation threshold indicates a larger range of disease areas that the neural network model tends to output.

[0072] Alternatively, for example, doctors can modify, supplement, or delete disease descriptions in disease areas through interface adjustments, directly modifying the disease descriptions corresponding to the disease areas.

[0073] Thus, based on the panoramic image of the oral cavity, the disease region and its description have been obtained. The obtained disease region and its description can be associated and stored, allowing for direct output when the panoramic image is loaded later.

[0074] The disease information is updated in real time after each adjustment. Based on the doctor's adjustments, the disease area on the panoramic oral image is updated, and the corresponding disease description is adjusted simultaneously. The results of the adjustments are automatically recorded and fed back to the neural network model as needed, so as to optimize the model in subsequent processing and improve the accuracy and stability of disease identification.

[0075] Through the aforementioned interactive adjustment operations, doctors can meticulously correct the automated recognition results of the neural network model, thereby improving the accuracy and adaptability of the neural network model in recognizing oral diseases. This flexible interactive adjustment operation is particularly suitable for complex cases or situations where the initial recognition is incomplete, ensuring that the final recognition result reflects the patient's true condition. This further enhances the flexibility and accuracy of the technical solution provided in this disclosure in practical applications, meeting the high standards required in medical practice.

[0076] In this embodiment, by outputting a panoramic image of the oral cavity, and then responding to user operations based on the panoramic image, information about a specified region on the panoramic image is obtained. This enables the use of a neural network model to output disease information on the panoramic image based on the panoramic image and the specified region information. Because the specified region information on the panoramic image is additionally provided based on user operations, the output results of the neural network model are more accurate, and the operation is simple and convenient, thereby improving the efficiency and reliability of intraoral disease identification.

[0077] In addition, by using the technical solution provided in this disclosure, doctors can provide relevant information about the problem area on the panoramic image of the oral cavity through simple interface interaction, which can effectively provide prompt information to the neural network model, thereby improving the reliability of the neural network model in recognizing oral diseases.

[0078] In addition, by using the technical solution provided in this disclosure, doctors can adjust the disease information on the panoramic oral image output by the neural network model, thereby enabling convenient and automated intervention of the neural network model. A large amount of doctor intervention data can be effectively applied to subsequent training of a more accurate neural network model.

[0079] In addition, the technical solutions provided in this disclosure can effectively improve the user experience.

[0080] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that this disclosure is not limited to the described order of actions, because according to this disclosure, some steps can be performed in other orders or simultaneously. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions and modules involved are not necessarily essential to this disclosure.

[0081] In the above embodiments, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions in other embodiments.

[0082] Figure 2 This is a schematic diagram based on the second embodiment of the present disclosure, as shown below. Figure 2 As shown. The oral panoramic image processing device 200 of this embodiment may include an image output unit 201, an information prompting unit 202, and a disease output unit 203. The image output unit 201 is used to output an oral panoramic image within the oral cavity; the information prompting unit 202 is used to obtain information about a specified region on the oral panoramic image in response to user operations based on the oral panoramic image; and the disease output unit 203 is used to output disease information on the oral panoramic image based on the oral panoramic image and the specified region information, using a neural network model.

[0083] The disease information on the panoramic oral image may include, but is not limited to, the following: The disease area on the panoramic image of the oral cavity, for example, the geometric features of the disease area such as coordinates, size, and shape; or The disease area on the panoramic image of the oral cavity and the corresponding disease description, such as the disease name (e.g., dental caries, alveolar bone resorption), the degree of the disease (e.g., mild, severe), and the formation of the disease.

[0084] It should be noted that some or all of the oral panoramic image processing device in this embodiment may be an application located on a local terminal, or it may be a plugin or software development kit (SDK) or other functional unit set in the application located on the local terminal, or it may be a processing engine located on the network-side server, or it may be a distributed system located on the network side, such as a processing engine or distributed system in a network-side oral panoramic image processing platform, etc. This embodiment does not impose any particular limitation on this.

[0085] It is understood that the application can be a native program installed on the local terminal, or it can be a web application of a browser on the local terminal. This embodiment does not limit this.

[0086] Optionally, in one possible implementation of this embodiment, the disease output unit 203 may further be used to freeze the weights of the encoder and the weights of the first decoder in the pre-trained segmentation network model; and to update the weights of the second decoder in the segmentation network model using a stochastic gradient descent algorithm to obtain the neural network model.

[0087] Optionally, in one possible implementation of this embodiment, the disease output unit 203 may be used to obtain disease information on the oral panoramic image based on the oral panoramic image and the specified region information using a neural network model; and to visualize and output the disease information on the oral panoramic image.

[0088] In a specific implementation, the disease output unit 203 can also be further used to adjust the disease information on the oral panoramic image in response to user operations based on the disease information on the oral panoramic image.

[0089] It should be noted that, Figure 1 The method in the corresponding embodiment can be implemented by the oral panoramic image processing device provided in this embodiment. For a detailed description, please refer to... Figure 1 The relevant content in the corresponding embodiments will not be repeated here.

[0090] In this embodiment, the image output unit outputs a panoramic image of the oral cavity. Then, the information prompting unit responds to user operations based on the panoramic image to obtain information about a specified region on the panoramic image. This enables the disease output unit to output disease information on the panoramic image based on the panoramic image and the specified region information, using a neural network model. Because the specified region information on the panoramic image is provided additionally based on user operations, the output results of the neural network model are more accurate and the operation is simple and convenient, thereby improving the efficiency and reliability of intraoral disease identification.

[0091] In addition, by using the technical solution provided in this disclosure, doctors can provide relevant information about the problem area on the panoramic image of the oral cavity through simple interface interaction, which can effectively provide prompt information to the neural network model, thereby improving the reliability of the neural network model in recognizing oral diseases.

[0092] In addition, by using the technical solution provided in this disclosure, doctors can adjust the disease information on the panoramic oral image output by the neural network model, thereby enabling convenient and automated intervention of the neural network model. A large amount of doctor intervention data can be effectively applied to subsequent training of a more accurate neural network model.

[0093] In addition, the technical solutions provided in this disclosure can effectively improve the user experience.

[0094] The acquisition, storage, and application of panoramic oral images within the oral cavity involved in this technical solution comply with relevant laws and regulations and do not violate public order and good morals.

[0095] According to embodiments of this disclosure, this disclosure also provides an electronic device, a readable storage medium, and a computer program product.

[0096] Figure 3 A schematic block diagram of an example electronic device 300 that can be used to implement embodiments of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workbenches, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital assistants, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0097] like Figure 3 As shown, the electronic device 300 includes a computing unit 301, which can perform various appropriate actions and processes based on a computer program stored in a read-only memory (ROM) 302 or a computer program loaded from a storage unit 308 into a random access memory (RAM) 303. The RAM 303 may also store various programs and data required for the operation of the electronic device 300. The computing unit 301, ROM 302, and RAM 303 are interconnected via a bus 304. An input / output (I / O) interface 305 is also connected to the bus 304.

[0098] Multiple components in electronic device 300 are connected to I / O interface 305, including: input unit 306, such as keyboard, mouse, etc.; output unit 307, such as various types of displays, speakers, etc.; storage unit 308, such as disk, optical disk, etc.; and communication unit 309, such as network card, modem, wireless transceiver, etc. Communication unit 309 allows electronic device 300 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0099] The computing unit 301 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 301 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 301 performs the various methods and processes described above, such as the method for processing panoramic oral images. For example, in some embodiments, the method for processing panoramic oral images can be implemented as a computer software program tangibly contained in a machine-readable medium, such as storage unit 308. In some embodiments, part or all of the computer program can be loaded and / or installed on the electronic device 300 via ROM 302 and / or communication unit 309. When the computer program is loaded into RAM 303 and executed by the computing unit 301, one or more steps of the method for processing panoramic oral images described above can be performed. Alternatively, in other embodiments, the computing unit 301 can be configured to perform the method for processing panoramic oral images by any other suitable means (e.g., by means of firmware).

[0100] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0101] The program code used to implement the methods of this disclosure may be written in any combination of one or more programming languages. This program code may be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing apparatus, such that when executed by the processor or controller, the program code causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code may be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0102] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0103] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device for displaying information to the user (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor); and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0104] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as data servers), or middleware components (e.g., application servers), or frontend components (e.g., user computers with graphical user interfaces or web browsers through which users can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., communication networks). Examples of communication networks include local area networks (LANs), wide area networks (WANs), the Internet, and blockchain networks.

[0105] Computer systems can include clients and servers. Clients and servers are generally geographically separated and typically interact via communication networks. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. A server can be a cloud server, also known as a cloud computing server or cloud host, a hosting product within the cloud computing service system that addresses the shortcomings of traditional physical hosts and VPS (Virtual Private Server) services, such as high management difficulty and weak business scalability. Servers can also be servers for distributed systems or servers incorporating blockchain technology.

[0106] It should be understood that the various forms of processes shown above can be used to rearrange, add, or delete steps. For example, the steps described in this disclosure can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution disclosed in this disclosure can be achieved, and this is not limited herein.

[0107] The specific embodiments described above do not constitute a limitation on the scope of protection of this disclosure. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this disclosure should be included within the scope of protection of this disclosure.

Claims

1. A method for processing panoramic images of the oral cavity, characterized in that, include: Output a panoramic image of the oral cavity; In response to a user operation based on the panoramic oral image, information about a specified region on the panoramic oral image is obtained; Based on the panoramic oral image and the information of the specified region, a neural network model is used to output disease information on the panoramic oral image.

2. The method according to claim 1, characterized in that, The disease information on the panoramic oral image includes: The disease area on the panoramic image of the oral cavity; or The disease area on the panoramic image of the oral cavity and the corresponding disease description on the panoramic image of the oral cavity.

3. The method according to claim 1, characterized in that, Before outputting disease information from the panoramic oral image using a neural network model based on the panoramic oral image and the specified region information, the method further includes: Freeze the encoder weights and the first part of the decoder weights in the pre-trained segmentation network model; The weights of the second decoder in the segmentation network model are updated using the stochastic gradient descent algorithm to obtain the neural network model.

4. The method according to any one of claims 1-3, characterized in that, The step of outputting disease information from the panoramic oral image based on the oral cavity image and the information of the specified region using a neural network model includes: Based on the panoramic oral image and the information of the specified region, a neural network model is used to obtain disease information on the panoramic oral image. The disease information on the panoramic image of the oral cavity is visualized and output.

5. The method according to claim 4, characterized in that, After visualizing and outputting the disease information on the panoramic oral image, the method further includes: In response to a user action based on disease information on the panoramic oral image, the disease information on the panoramic oral image is adjusted.

6. A device for processing panoramic oral images, characterized in that, include: Image output unit, used to output panoramic images of the oral cavity; Information prompting unit is used to obtain information about a specified area on the oral panoramic image in response to user operation based on the oral panoramic image; The disease output unit is used to output disease information on the oral panoramic image based on the oral panoramic image and the information of the specified region, using a neural network model.

7. The apparatus according to claim 6, characterized in that, The disease information on the panoramic oral image includes at least one of the following: The disease area on the panoramic image of the oral cavity; or The disease area on the panoramic image of the oral cavity and the corresponding disease description on the panoramic image of the oral cavity.

8. The apparatus according to claim 6, characterized in that, The disease output unit is also used for Freeze the encoder weights and the first part of the decoder weights in the pre-trained segmentation network model; as well as The weights of the second decoder in the segmentation network model are updated using the stochastic gradient descent algorithm to obtain the neural network model.

9. The apparatus according to any one of claims 6-8, characterized in that, The disease output unit is specifically used for Based on the panoramic oral image and the information of the designated region, a neural network model is used to obtain disease information on the panoramic oral image; and The disease information on the panoramic image of the oral cavity is visualized and output.

10. The apparatus according to claim 9, characterized in that, The disease output unit is also used for In response to a user action based on disease information on the panoramic oral image, the disease information on the panoramic oral image is adjusted.

11. An electronic device, characterized in that, include: At least one processor; as well as A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1-5.

12. A non-transitory computer-readable storage medium storing computer instructions, characterized in that, The computer instructions are used to cause the computer to perform the method according to any one of claims 1-5.

13. A computer program product, characterized in that, Includes a computer program that, when executed by a processor, implements the method according to any one of claims 1-5.

Images