Image generation method and system based on oral cavity imaging

Abstract

Disclosed in the present invention are an image generation method and system based on oral cavity imaging. The method comprises: acquiring CT image data of an oral cavity to be processed, and constructing an oral cavity skeleton model on the basis of the CT image data; extracting skeleton features of the oral cavity skeleton model, and, by means of a preset healthy soft and hard tissue oral cavity model database and on the basis of the skeleton features, covering the oral cavity skeleton model with soft tissue to obtain a first oral cavity model; performing three-dimensional laser scanning on said oral cavity so as to construct a second oral cavity model, comparing the first oral cavity model with the second oral cavity model to obtain a soft tissue difference area of the first oral cavity model and the second oral cavity model, and marking the soft tissue difference area in the second oral cavity model; acquiring optical image information of a target acquisition area in said oral cavity, and performing soft tissue feature extraction on the optical image information; and covering the second oral cavity model with extracted soft tissue features to obtain a three-dimensional image model of said oral cavity, and then performing oral image acquisition on the three-dimensional image model so as to generate an oral cavity image.

Description

Image generation method and system based on oral imaging Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to an image generation method and system based on oral imaging. Background Art

[0002] Oral examination and treatment are an essential part of the treatment of head and neck tumors, especially oral diagnosis and treatment before radiotherapy of head and neck tumors and subsequent follow-up examinations, as well as the diagnosis and treatment of oral complications during anti-tumor treatment. All of these require imaging analysis of the patient's crowns, roots, periapical tissues, and jaw bones to better assist doctors in diagnosing oral diseases and determining treatment plans.

[0003] Currently, intraoral imaging typically utilizes an image sensor equipped with X-ray recognition or oral CBCT, coupled with a digital processing output circuit to transmit digital signals to a computer, where application software displays the two-dimensional or three-dimensional image data. However, existing X-ray recognition image sensors or oral CT can only capture oral hard tissue structures such as teeth and jawbone. X-rays and oral CT have strong penetration into soft tissue, making it difficult to accurately identify oral muscles and / or tumor-related soft tissue. Optical imaging of the oral cavity is susceptible to the complex three-dimensional structure of the oral cavity, resulting in a limited field of view, tissue occlusion, and insufficient illumination. Ultimately, this makes it impossible to capture clear and accurate images of the complete intraoral tissue structure. Subsequently, muscle tissue images must be captured and stitched together from various angles. However, these stitched images only display two-dimensional oral information and cannot accurately and completely display the overall soft tissue structure, resulting in an oral image that fails to truly reproduce the patient's actual oral condition.

[0004] Therefore, there is an urgent need for a method that can improve the accuracy, clarity and restoration of oral imaging images.

[0005] Summary of the Invention

[0006] The present invention provides an image generation method and system based on oral imaging to solve the technical problems of low clarity, low accuracy and low restoration of oral imaging in the prior art.

[0007] In order to solve the above technical problems, an embodiment of the present invention provides an image generation method based on oral imaging, comprising:

[0008] Acquiring CT image data of the oral cavity to be processed, and constructing an oral skeletal model based on the CT image data;

[0009] Extracting skeletal features of the oral skeletal model, and overlaying soft tissue on the oral skeletal model according to the skeletal features using a preset database of healthy soft and hard tissue oral models to obtain a first oral model; wherein the preset database of healthy soft and hard tissue oral models stores oral models containing healthy soft and hard tissues;

[0010] Performing a three-dimensional laser scan of the oral cavity to be treated to construct a second oral cavity model, and comparing the first oral cavity model with the second oral cavity model to obtain soft tissue difference areas between the two, and marking the areas in the second oral cavity model;

[0011] Collecting optical image information of a target acquisition area in the oral cavity to be processed, and performing soft tissue feature extraction on the optical image information; wherein the target acquisition area corresponds to the soft tissue difference area marked on the second oral cavity model;

[0012] The extracted soft tissue features are overlaid on the second oral cavity model to obtain a three-dimensional image model of the oral cavity to be processed, and then the oral cavity image is collected from the three-dimensional image model to generate an oral cavity image.

[0013] As a preferred solution, the method of collecting CT image data of the oral cavity to be processed and constructing an oral skeletal model based on the CT image data is specifically as follows:

[0014] The cone beam imaging device scans the oral cavity layer by layer according to the preset scanning interval to obtain the CT image data corresponding to each layer;

[0015] Perform bone hard tissue recognition on the CT image data corresponding to each layer, and extract the hard tissue feature model data corresponding to each CT image data;

[0016] sequentially correcting hard tissue feature model data extracted from CT image data corresponding to each layer, so that each time the hard tissue feature model data of one layer is corrected, the hard tissue feature model data adjacent to the layer is used as corrected reference model data, and the corrected reference model data is combined with the hard tissue feature model data of the layer, thereby calculating a first edge smoothness between the combined corrected reference model data and the hard tissue feature model data of the layer, and then correcting the hard tissue feature model data of the layer based on the first edge smoothness;

[0017] The corrected hard tissue feature model data of each layer are combined and merged to construct an oral bone model.

[0018] As a preferred solution, the hard tissue feature model data of the layer is corrected according to the first edge smoothness, specifically:

[0019] When the first edge smoothness is within a preset threshold range, there is no need to modify the hard tissue feature model data of the layer;

[0020] When the first edge smoothness is outside a preset threshold range, a second edge smoothness is calculated between the hard tissue feature model data of each layer in the corrected reference model data, and when the difference between the first edge smoothness and the second edge smoothness is greater than a preset error, smoothing correction is performed on the edge of the hard tissue feature model data of the layer according to the second edge smoothness, thereby completing the correction of the hard tissue feature model data of the layer;

[0021] Until the hard tissue feature model data of each layer is corrected.

[0022] As a preferred solution, the bone features of the oral skeletal model are extracted, and the oral skeletal model is covered with soft tissue according to the bone features through a preset healthy soft and hard tissue oral model database to obtain a first oral model, specifically:

[0023] Extracting bone features of preset key areas in the oral skeleton model; wherein the preset key areas include the maxillary area, the mandibular area, the maxillary alveolar bone area, and the mandibular alveolar bone area;

[0024] Retrieving a soft and hard tissue oral cavity model with the greatest similarity to the skeletal features of the preset key area from a preset healthy soft and hard tissue oral cavity model database as a reference model; wherein the healthy soft and hard tissue oral cavity model database pre-stores several soft and hard tissue oral cavity models corresponding to the skeletal features of different preset key areas;

[0025] Soft tissue data of the reference model is extracted, and the soft tissue data is overlaid on the oral skeletal model to construct a first oral model.

[0026] As a preferred solution, the oral cavity to be treated is subjected to a three-dimensional laser scan to construct a second oral cavity model, and the first oral cavity model and the second oral cavity model are compared to obtain the soft tissue difference areas between the two, and mark them in the second oral cavity model, specifically:

[0027] Performing a three-dimensional laser scan on the oral cavity to be treated to obtain an oral structure model;

[0028] The oral skeletal model is matched with the oral structure model to construct a second oral model;

[0029] Comparing the first oral cavity model with the second oral cavity model, and sequentially obtaining regions where there are structural differences between the first oral cavity model and the second oral cavity model;

[0030] If the region of the difference structure exists in the first oral cavity model, the region of the difference structure is eliminated;

[0031] If the region of the difference structure exists in the second oral cavity model, the region of the difference structure is regarded as a soft tissue difference region and marked in the second oral cavity model.

[0032] As a preferred solution, the optical image information of the target acquisition area in the oral cavity to be processed is collected, and soft tissue feature extraction is performed on the optical image information, specifically:

[0033] Collecting optical image information of a target acquisition area in the oral cavity to be processed, and sequentially performing pathological identification on the optical image information according to the pathological identification model; wherein the target acquisition area corresponds to the soft tissue difference area marked on the second oral cavity model;

[0034] Soft tissue feature recognition is performed on the optical image information with identified pathological features, thereby extracting soft tissue feature data with pathological features.

[0035] As a preferred solution, the extracted soft tissue features are overlaid onto the second oral cavity model to obtain a three-dimensional image model of the oral cavity to be processed, specifically:

[0036] According to the target acquisition area of ​​the optical image information in which the pathological features are identified, the soft tissue feature data extracted from the pathological features are sequentially overlaid on the second oral model in the corresponding area;

[0037] The soft tissue difference areas corresponding to the optical image information in which no physical features are identified in the second oral cavity model are marked and removed to obtain the final second oral cavity model;

[0038] According to the soft tissue data in the first oral cavity model, the unlabeled areas in the final second oral cavity model are covered with soft tissue, thereby obtaining a three-dimensional image model of the oral cavity to be processed.

[0039] Accordingly, the present invention also provides an image generation system based on oral imaging, comprising: an acquisition device and a processing host; the acquisition device comprises: a cone beam imaging device, a laser three-dimensional scanning device and an optical image acquisition device;

[0040] The processing host includes: a CT image module, a first oral cavity model module, a second oral cavity model module, a soft tissue feature module and a soft tissue coverage module;

[0041] The CT image module is used to collect CT image data of the oral cavity to be processed and construct an oral skeletal model based on the CT image data;

[0042] The first oral model module is used to extract the skeletal features of the oral skeletal model and, using a preset database of healthy soft and hard tissue oral models, overlay the oral skeletal model with soft tissue according to the skeletal features to obtain a first oral model; wherein the preset database of healthy soft and hard tissue oral models stores oral models containing healthy soft and hard tissues;

[0043] The second oral cavity model module is used to perform a three-dimensional laser scan of the oral cavity to be processed, thereby constructing a second oral cavity model, and comparing the first oral cavity model with the second oral cavity model to obtain soft tissue difference areas between the two, and mark them in the second oral cavity model;

[0044] The soft tissue feature module is configured to collect optical image information of a target collection area in the oral cavity to be processed and perform soft tissue feature extraction on the optical image information; wherein the target collection area corresponds to the soft tissue difference area marked on the second oral cavity model;

[0045] The soft tissue covering module is used to cover the extracted soft tissue features into the second oral cavity model to obtain a three-dimensional image model of the oral cavity to be processed, and then collect an oral image of the three-dimensional image model to generate an oral image.

[0046] Accordingly, the present invention also provides a terminal device, comprising a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, wherein the processor implements the image generation method based on oral imaging as described in any one of the above items when executing the computer program.

[0047] Accordingly, the present invention also provides a computer-readable storage medium, which includes a stored computer program; wherein, when the computer program is running, it controls the device where the computer-readable storage medium is located to execute the image generation method based on oral imaging as described in any one of the above items.

[0048] Compared with the prior art, the embodiments of the present invention have the following beneficial effects:

[0049] The technical solution of the present invention compares the oral CT image data covering the soft tissue with the second oral model scanned by laser three-dimensional scanning, thereby obtaining the soft tissue difference area between the two, and combines the optical image information to cover the actual feature data of the soft tissue in the soft tissue difference area, thereby accurately obtaining a three-dimensional image model of the oral cavity to be processed, and then generating the corresponding oral image through the three-dimensional image model, avoiding the existing collection and analysis of pathological soft tissues such as muscles and / or tumors only through optical images, resulting in the need for image splicing in the case of multiple optical image data, which in turn leads to easy image distortion. At the same time, the two-dimensional optical image cannot provide specific structural information of pathological soft tissues such as muscles and / or tumors, while the three-dimensional image model of the present invention can accurately and clearly restore the structural information of the oral cavity to be processed, so that the present invention has a high degree of restoration, which facilitates doctors to analyze the pathological conditions of the oral cavity. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] FIG1 is a flowchart of the steps of an image generation method based on oral imaging provided by an embodiment of the present invention;

[0051] FIG2 is a schematic diagram of hard tissue feature model data of each layer provided by an embodiment of the present invention;

[0052] FIG3 is a schematic structural diagram of an image generation system based on oral imaging provided by an embodiment of the present invention;

[0053] FIG4 is a schematic structural diagram of a processing host provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0054] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts shall fall within the scope of protection of the present invention.

[0055] In the first embodiment, referring to FIG. 1 , an image generation method based on oral imaging is provided in an embodiment of the present invention, comprising the following steps S101 to S105:

[0056] Step S101: Acquire CT image data of the oral cavity to be processed, and construct an oral skeletal model based on the CT image data.

[0057] It should be noted that the embodiment of the present invention is mainly used for oral image processing of soft tissue abnormalities in the oral cavity, rather than oral image processing of hard tissues such as bones. Therefore, the image generation method based on oral imaging in the embodiment of the present invention is mainly used for cases of soft tissue abnormalities such as tumors and / or muscles.

[0058] As a preferred solution, the method of collecting CT image data of the oral cavity to be processed and constructing an oral skeletal model based on the CT image data is specifically as follows:

[0059] Through the cone beam imaging device, the oral cavity to be processed is scanned layer by layer according to the preset scanning interval to obtain the CT image data corresponding to each layer; the bone hard tissue is identified on the CT image data corresponding to each layer, and the hard tissue feature model data corresponding to each CT image data is extracted; the hard tissue feature model data extracted from the CT image data corresponding to each layer are corrected in turn, so that in the process of correcting the hard tissue feature model data of one layer each time, the hard tissue feature model data adjacent to the layer is used as the corrected reference model data, and the corrected reference model data is combined with the hard tissue feature model data of the layer, so as to calculate the first edge smoothness between the combined corrected reference model data and the hard tissue feature model data of the layer, and then the hard tissue feature model data of the layer is corrected according to the first edge smoothness; the hard tissue feature model data of each layer after correction are combined and merged to construct an oral bone model.

[0060] In this embodiment, the cone-beam imaging device can scan the oral cavity to be treated layer by layer, thereby obtaining CT image data corresponding to each layer. The scanning interval can be set based on the scanning penetration capability of the cone-beam imaging device and the actual imaging requirements of the oral cavity to be treated. The scanning interval refers to the distance between each layer of CT image data.

[0061] Furthermore, it can be understood that scanning the oral cavity to be treated layer by layer can be understood as acquiring the oral cavity cross-section image layer by layer at a certain distance (scanning interval) of the oral cavity to be treated, and the CT image data of each layer is the cross-section image of the oral cavity to be treated.

[0062] In this embodiment, bone hard tissue recognition can be achieved for each layer of CT image data by pre-training a model that can recognize bone data from CT image data. That is, the bone hard tissue recognition model is trained using a large number of training sets and test sets of CT image data containing bone information, and the bone hard tissue of each layer of CT image data corresponding to the CT image data can be quickly and accurately identified, thereby extracting the hard tissue feature model data corresponding to each CT image data.

[0063] In this embodiment, the hard tissue feature model data extracted from the CT image data corresponding to each layer is corrected in sequence, so that each time the hard tissue feature model data of one layer is corrected, the hard tissue feature model data of two adjacent layers or one layer of the layer is used as the correction reference model data. Then, by merging the corrected reference model data with the hard tissue feature model data of the current layer, the edge smoothness between the hard tissue feature model data of two layers in the corrected reference model data and the hard tissue feature model data of the current layer (a total of three layers of hard tissue feature model data) can be obtained, as specifically shown in FIG2 ; or, the edge smoothness between the hard tissue feature model data of one layer in the corrected reference model data and the hard tissue feature model data of the current layer can be obtained. In this case, this only occurs when the current layer is the first layer or the last layer in all the data.

[0064] In this embodiment, edge smoothness can be calculated by the distance between edge data points at corresponding positions in each layer. Since CT image data is acquired through the same cone-beam imaging device, the CT image data of each layer are all in the same coordinate system, and the position information of the data points of the hard tissue feature model data in each layer can be assigned, so that the edge smoothness between each layer can be calculated accurately and quickly.

[0065] As a preferred solution, the hard tissue feature model data of the layer is corrected according to the first edge smoothness, specifically:

[0066] When the first edge smoothness is within a preset threshold range, there is no need to correct the hard tissue feature model data of the layer; when the first edge smoothness is outside the preset threshold range, the second edge smoothness between the hard tissue feature model data of each layer in the corrected reference model data is calculated, and when the difference between the first edge smoothness and the second edge smoothness is greater than a preset error, the edge of the hard tissue feature model data of the layer is smoothed and corrected according to the second edge smoothness, thereby completing the correction of the hard tissue feature model data of the layer; until the hard tissue feature model data of each layer is corrected.

[0067] In this embodiment, when the first edge smoothness is within the preset threshold range, it indicates that the hard tissue feature model data of each layer in the corrected reference model data and the hard tissue feature model data of the current layer are consistent with the skeletal conditions of the normal oral cavity, and therefore there is no need to correct the hard tissue feature model data of the layer.

[0068] In this embodiment, when the first edge smoothness is outside the preset threshold range, it is clear that the hard tissue feature model data of each layer in the corrected reference model data and the hard tissue feature model data of the current layer do not conform to the skeletal conditions of the normal oral cavity. Therefore, it is necessary to calculate the second edge smoothness between the hard tissue feature model data of the two layers in the corrected reference model data to smooth the edges of the hard tissue feature model data of the layer. Furthermore, the smoothing correction of the edges of the hard tissue feature model data of the layer can be performed by taking the average data point between the hard tissue feature model data in the corrected reference model data as the reference value for the correction amount for smoothing the edges of the hard tissue feature model data of the layer. At the same time, the CT image data of the oral cavity to be processed can be re-acquired and recalculated to ensure that the error is not caused by factors of the acquisition equipment.

[0069] Furthermore, when the first edge smoothness of the re-collected CT image data is still outside the preset threshold range, it indicates that there is still an abnormality in the hard tissue of the oral cavity to be processed, and a corresponding message is generated for alarm output.

[0070] Furthermore, for the case where the current layer is the first layer or the last layer among all the data, since it is the starting layer or the ending layer, its hard tissue feature model data in the CT image data is a data feature point. Therefore, the first layer or the last layer CT image data will not have a first edge smoothness outside the preset threshold range.

[0071] Step S102: extracting the skeletal features of the oral skeletal model, and covering the oral skeletal model with soft tissue according to the skeletal features through a preset healthy soft and hard tissue oral model database to obtain a first oral model; wherein the preset healthy soft and hard tissue oral model database stores an oral model containing healthy soft and hard tissues.

[0072] As a preferred solution, the bone features of the oral skeletal model are extracted, and the oral skeletal model is covered with soft tissue according to the bone features through a preset healthy soft and hard tissue oral model database to obtain a first oral model, specifically:

[0073] Extract the skeletal features of preset key areas in the oral skeletal model; wherein the preset key areas include the maxillary area, the mandibular area, the maxillary alveolar bone area and the mandibular alveolar bone area; in the preset healthy soft and hard tissue oral model database, retrieve the soft and hard tissue oral model with the greatest similarity to the skeletal features of the preset key areas as a reference model; wherein the healthy soft and hard tissue oral model database pre-stores several soft and hard tissue oral models corresponding to the skeletal features of different preset key areas; extract the soft tissue data of the reference model, and overlay the soft tissue data on the oral skeletal model to construct a first oral model.

[0074] In this embodiment, the bone features of the preset key areas in the oral bone model can be quickly and accurately extracted through the pre-trained bone recognition model. The bone features are bone structure features that can reflect the details of the bone structure, including the distance between the feature points and the degree of curvature during the bone recognition process. Furthermore, by extracting the bone features of the preset key areas in the oral bone model, the soft and hard tissue oral model with the greatest similarity to the bone features of the preset key areas can be retrieved from the preset healthy soft and hard tissue oral model database to serve as a reference model for soft tissue coverage. The soft and hard tissue oral model includes soft tissue data and hard tissue data (including bones), so the corresponding soft and hard tissue oral model can be determined through the bone feature data, and then the soft tissue in the determined soft and hard tissue oral model is covered in the oral bone model, thereby ensuring the accuracy and feasibility of the comparison model used as a standard healthy soft tissue reference in the first oral model.

[0075] In this embodiment, the first oral cavity model can be constructed quickly and accurately by extracting the soft tissue data of the reference model and overlaying the soft tissue data on the oral cavity skeleton model.

[0076] Step S103: performing a three-dimensional laser scan on the oral cavity to be processed to construct a second oral cavity model, and comparing the first oral cavity model with the second oral cavity model to obtain soft tissue difference areas between the two, and marking them in the second oral cavity model.

[0077] As a preferred solution, the oral cavity to be treated is subjected to a three-dimensional laser scan to construct a second oral cavity model, and the first oral cavity model and the second oral cavity model are compared to obtain the soft tissue difference areas between the two, and mark them in the second oral cavity model, specifically:

[0078] Perform a three-dimensional laser scan on the oral cavity to be processed to obtain an oral structure model; combine the oral skeletal model with the oral structure model to construct a second oral model; compare the first oral model with the second oral model to obtain areas with different structures between the first oral model and the second oral model in turn; if the area with different structures exists in the first oral model, eliminate the area with different structures; if the area with different structures exists in the second oral model, treat the area with different structures as a soft tissue difference area and mark it in the second oral model.

[0079] In this embodiment, an oral structure model is obtained by performing a three-dimensional laser scan of the oral cavity to be processed. It is understood that since the principle of three-dimensional laser scanning is to construct an oral structure model by emitting a laser and receiving the reflected laser information, this oral structure model cannot accurately represent the specific pathological conditions of soft tissues. It can only reflect the specific oral structure (including the size, structure, and location of the soft tissues), but cannot reflect the condition of the soft tissues.

[0080] In this embodiment, the oral structure model obtained can be used to match the oral skeletal model with the oral structure model. Since the oral structure model represents the specific structural information of the oral cavity to be processed, the skeletal data in the oral skeletal model can accurately ensure the structural integrity of the oral structure model. When the oral skeletal model is matched with the oral structure model, if some of the oral skeletal model is exposed, it means that the oral skeletal model and the oral structure model are not based on the data of the same oral cavity to be processed. There may be a problem with the equipment or the acquisition process, which requires manual confirmation by the equipment operator. Then, a second oral model can be constructed by matching them.

[0081] In this embodiment, by comparing the second oral model with the first oral model, we can find areas where there are differences in soft tissue structure (including: soft tissue size and position) between the two. However, when the area of ​​the difference structure exists in the first oral model, since the first oral model is based on data information of a healthy oral cavity, the difference structure may be a problem of missing oral soft tissue to be treated and is not within the scope of consideration for tumor detection and diagnosis. That is, the area of ​​the difference structure needs to be eliminated. The difference structure existing in the second oral model is the soft tissue structure of the oral soft tissue to be treated compared to that in a healthy oral cavity, which may indicate oral soft tissue abnormality. Therefore, the area of ​​the difference structure needs to be treated as a soft tissue difference area and marked in the second oral model.

[0082] Step S104: collecting optical image information of a target collection area in the oral cavity to be processed, and performing soft tissue feature extraction on the optical image information; wherein the target collection area corresponds to the soft tissue difference area marked on the second oral cavity model.

[0083] As a preferred solution, the optical image information of the target acquisition area in the oral cavity to be processed is collected, and soft tissue feature extraction is performed on the optical image information, specifically:

[0084] Optical image information of a target acquisition area in the oral cavity to be processed is collected, and pathological identification is performed on the optical image information in sequence according to a pathological identification model; wherein the target acquisition area corresponds to a soft tissue difference area marked on the second oral cavity model; soft tissue feature identification is performed on the optical image information in which pathological features are identified, thereby extracting soft tissue feature data containing pathological features.

[0085] In this embodiment, by collecting optical image information of the target collection area in the oral cavity to be processed, the soft tissue surface pathological structure information corresponding to the soft tissue difference area on the second oral model can be obtained, so that the optical image information with identified pathological characteristics can be accurately used to identify soft tissue characteristics and extract soft tissue feature data with pathological characteristics.

[0086] Step S105: Overlaying the extracted soft tissue features onto the second oral cavity model to obtain a three-dimensional image model of the oral cavity to be processed, and then collecting an oral cavity image from the three-dimensional image model to generate an oral cavity image.

[0087] As a preferred solution, the extracted soft tissue features are overlaid onto the second oral cavity model to obtain a three-dimensional image model of the oral cavity to be processed, specifically:

[0088] According to the target acquisition area of ​​the optical image information with identified pathological characteristics, the soft tissue feature data with pathological characteristics are extracted and covered in the second oral model of the corresponding area in sequence; the soft tissue difference areas corresponding to the optical image information with no identified pathological characteristics in the second oral model are marked and eliminated to obtain the final second oral model; according to the soft tissue data in the first oral model, the unmarked areas in the final second oral model are covered with soft tissue to obtain a three-dimensional image model of the oral cavity to be processed.

[0089] In this embodiment, based on the target acquisition area of ​​the optical image information that identifies the pathological characteristics, the extracted soft tissue feature data with pathological characteristics can be sequentially covered in the second oral model of the corresponding area, that is, the second oral model completes the coverage of the soft tissue feature data of the marked area that may have pathological characteristics, so that the second oral model can display the soft tissue structure that may have pathological characteristics in three dimensions.

[0090] Furthermore, the soft tissue difference areas corresponding to the optical image information of the second oral model in which the physical features are not identified are marked and eliminated to obtain the final second oral model, thereby ensuring that the other unmarked areas of the second oral model can be covered by soft tissue, thereby ensuring that the final three-dimensional image model of the oral cavity to be treated can restore the oral cavity to be treated to the greatest extent, and the oral cavity to be treated can be used as a three-dimensional digital model to be displayed in a computer, which facilitates the doctor's pathological analysis of the oral cavity to be treated, and also avoids the situation where insufficient lighting points make it impossible to collect clear and accurate images of the complete oral cavity.

[0091] In this embodiment, by generating a corresponding three-dimensional image model, the problem of blurred and low accuracy of oral images due to insufficient lighting points can be avoided. At the same time, the three-dimensional image model can be reused multiple times without the need to collect multiple oral images as required previously. This improves the accuracy of oral image generation, increases the convenience for doctors to obtain oral images, and improves the user experience.

[0092] The implementation of the above embodiment has the following effects:

[0093] The technical solution of the present invention compares the oral CT image data covering the soft tissue with the second oral model scanned by laser three-dimensional scanning, thereby obtaining the soft tissue difference area between the two, and combines the optical image information to cover the actual feature data of the soft tissue in the soft tissue difference area, thereby accurately obtaining a three-dimensional image model of the oral cavity to be processed, and then generating the corresponding oral image through the three-dimensional image model, avoiding the existing collection and analysis of pathological soft tissues such as muscles and / or tumors only through optical images, resulting in the need for image splicing in the case of multiple optical image data, which in turn leads to easy image distortion. At the same time, the two-dimensional optical image cannot provide specific structural information of pathological soft tissues such as muscles and / or tumors, while the three-dimensional image model of the present invention can accurately and clearly restore the structural information of the oral cavity to be processed, so that the present invention has a high degree of restoration, which facilitates doctors to analyze the pathological conditions of the oral cavity.

[0094] Example 2, please refer to Figures 3 and 4, which show that the present invention also provides an image generation system based on oral imaging, including: an acquisition device 01 and a processing host 02; the acquisition device includes: a cone beam imaging device 011, a laser three-dimensional scanning device 012 and an optical image acquisition device 013.

[0095] It should be noted that cone-beam imaging equipment 011 (cone-beam CT) is the most promising and practical device among current oral and cranial imaging devices. 3D laser scanning equipment 012 (3D LASER scanner) is used to detect and analyze the shape (geometry) and appearance data (such as color, surface albedo, and other properties) of objects or environments in the real world. As the primary component of a 3D laser scanning system, a 3D laser scanner consists of a laser emitter, a receiver, a time counter, a motor-controlled rotatable filter, and a control circuit board. Optical image acquisition equipment 013 includes, but is not limited to, optical cameras, optical image detection sensors, and optical image acquisition devices.

[0096] The processing host 02 includes: a CT image module 201 , a first oral cavity model module 202 , a second oral cavity model module 203 , a soft tissue feature module 204 and a soft tissue coverage module 205 .

[0097] The CT image module 201 is used to collect CT image data of the oral cavity to be processed and construct an oral skeletal model based on the CT image data.

[0098] The first oral model module 202 is used to extract the bone features of the oral bone model, and through a preset healthy soft and hard tissue oral model database, cover the oral bone model with soft tissue according to the bone features to obtain a first oral model; wherein the preset healthy soft and hard tissue oral model database stores an oral model containing healthy soft and hard tissues.

[0099] The second oral cavity model module 203 is used to perform a three-dimensional laser scan of the oral cavity to be processed, thereby constructing a second oral cavity model, and comparing the first oral cavity model with the second oral cavity model to obtain the soft tissue difference areas between the two, and mark them in the second oral cavity model.

[0100] The soft tissue feature module 204 is used to collect optical image information of a target collection area in the oral cavity to be processed and perform soft tissue feature extraction on the optical image information; wherein the target collection area corresponds to the soft tissue difference area marked on the second oral cavity model.

[0101] The soft tissue covering module 205 is used to cover the extracted soft tissue features into the second oral cavity model to obtain a three-dimensional image model of the oral cavity to be processed, and then collect an oral image of the three-dimensional image model to generate an oral image.

[0102] As a preferred solution, the method of collecting CT image data of the oral cavity to be processed and constructing an oral skeletal model based on the CT image data is specifically as follows:

[0103] The cone beam imaging device scans the oral cavity layer by layer according to the preset scanning interval to obtain the CT image data corresponding to each layer;

[0104] Perform bone hard tissue recognition on the CT image data corresponding to each layer, and extract the hard tissue feature model data corresponding to each CT image data;

[0105] sequentially correcting the hard tissue feature model data extracted from the CT image data corresponding to each layer, so that each time the hard tissue feature model data of one layer is corrected, the hard tissue feature model data adjacent to the layer is used as corrected reference model data, and the corrected reference model data is combined with the hard tissue feature model data of the layer, thereby calculating a first edge smoothness between the combined corrected reference model data and the hard tissue feature model data of the layer, and then correcting the hard tissue feature model data of the layer based on the first edge smoothness;

[0106] The corrected hard tissue feature model data of each layer are combined and merged to construct an oral bone model.

[0107] As a preferred solution, the hard tissue feature model data of the layer is corrected according to the first edge smoothness, specifically:

[0108] When the first edge smoothness is within a preset threshold range, there is no need to modify the hard tissue feature model data of the layer;

[0109] When the first edge smoothness is outside a preset threshold range, a second edge smoothness is calculated between the hard tissue feature model data of each layer in the corrected reference model data, and when the difference between the first edge smoothness and the second edge smoothness is greater than a preset error, smoothing correction is performed on the edge of the hard tissue feature model data of the layer according to the second edge smoothness, thereby completing the correction of the hard tissue feature model data of the layer;

[0110] Until the hard tissue feature model data of each layer is corrected.

[0111] As a preferred solution, the bone features of the oral skeletal model are extracted, and the oral skeletal model is covered with soft tissue according to the bone features through a preset healthy soft and hard tissue oral model database to obtain a first oral model, specifically:

[0112] Extracting bone features of preset key areas in the oral skeleton model; wherein the preset key areas include the maxillary area, the mandibular area, the maxillary alveolar bone area, and the mandibular alveolar bone area;

[0113] Retrieving a soft and hard tissue oral cavity model with the greatest similarity to the skeletal features of the preset key area from a preset healthy soft and hard tissue oral cavity model database as a reference model; wherein the healthy soft and hard tissue oral cavity model database pre-stores several soft and hard tissue oral cavity models corresponding to the skeletal features of different preset key areas;

[0114] Soft tissue data of the reference model is extracted, and the soft tissue data is overlaid on the oral skeletal model to construct a first oral model.

[0115] As a preferred solution, the oral cavity to be treated is subjected to a three-dimensional laser scan to construct a second oral cavity model, and the first oral cavity model and the second oral cavity model are compared to obtain the soft tissue difference areas between the two, and mark them in the second oral cavity model, specifically:

[0116] Performing a three-dimensional laser scan on the oral cavity to be treated to obtain an oral structure model;

[0117] The oral skeletal model is matched with the oral structure model to construct a second oral model;

[0118] Comparing the first oral cavity model with the second oral cavity model, and sequentially obtaining regions where there are structural differences between the first oral cavity model and the second oral cavity model;

[0119] If the region of the difference structure exists in the first oral cavity model, the region of the difference structure is eliminated;

[0120] If the region of the difference structure exists in the second oral cavity model, the region of the difference structure is regarded as a soft tissue difference region and marked in the second oral cavity model.

[0121] As a preferred solution, the optical image information of the target acquisition area in the oral cavity to be processed is collected, and soft tissue feature extraction is performed on the optical image information, specifically:

[0122] Collecting optical image information of a target acquisition area in the oral cavity to be processed, and sequentially performing pathology identification on the optical image information according to the pathology identification model; wherein the target acquisition area corresponds to the soft tissue difference area marked on the second oral cavity model;

[0123] Soft tissue feature recognition is performed on the optical image information with identified pathological features, thereby extracting soft tissue feature data with pathological features.

[0124] As a preferred solution, the extracted soft tissue features are overlaid onto the second oral cavity model to obtain a three-dimensional image model of the oral cavity to be processed, specifically:

[0125] According to the target acquisition area of ​​the optical image information in which the pathological features are identified, the soft tissue feature data extracted from the pathological features are sequentially overlaid on the second oral model in the corresponding area;

[0126] The soft tissue difference areas corresponding to the optical image information in which no physical features are identified in the second oral cavity model are marked and removed to obtain the final second oral cavity model;

[0127] According to the soft tissue data in the first oral cavity model, the unlabeled areas in the final second oral cavity model are covered with soft tissue, thereby obtaining a three-dimensional image model of the oral cavity to be processed.

[0128] Those skilled in the art will clearly understand that, for the convenience and brevity of description, the specific working process of the system described above can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here.

[0129] The implementation of the present invention has the following effects:

[0130] The technical solution of the present invention compares the oral CT image data covering the soft tissue with the second oral model scanned by laser three-dimensional scanning, thereby obtaining the soft tissue difference area between the two, and combines the optical image information to cover the actual feature data of the soft tissue in the soft tissue difference area, thereby accurately obtaining a three-dimensional image model of the oral cavity to be processed, and then generating the corresponding oral image through the three-dimensional image model, avoiding the existing collection and analysis of pathological soft tissues such as muscles and / or tumors only through optical images, resulting in the need for image splicing in the case of multiple optical image data, which in turn leads to easy image distortion. At the same time, the two-dimensional optical image cannot provide specific structural information of pathological soft tissues such as muscles and / or tumors, while the three-dimensional image model of the present invention can accurately and clearly restore the structural information of the oral cavity to be processed, so that the present invention has a high degree of restoration, which facilitates doctors to analyze the pathological conditions of the oral cavity.

[0131] Embodiment 3. Accordingly, the present invention also provides a terminal device, comprising: a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, wherein when the processor executes the computer program, the image generation method based on oral imaging as described in any one of the above embodiments is implemented.

[0132] The terminal device of this embodiment includes: a processor, a memory, and a computer program or computer instructions stored in the memory and executable on the processor. When the processor executes the computer program, it implements the steps of the first embodiment described above, such as steps S101 to S105 shown in FIG1 . Alternatively, when the processor executes the computer program, it implements the functions of the modules / units of the system embodiment described above, such as the second oral cavity model module 203.

[0133] Exemplarily, the computer program can be divided into one or more modules / units, and the one or more modules / units are stored in the memory and executed by the processor to complete the present invention. The one or more modules / units can be a series of computer program instruction segments that can perform specific functions, and the instruction segments are used to describe the execution process of the computer program in the terminal device. For example, the second oral model module 203 is used to perform a three-dimensional laser scan of the oral cavity to be processed, thereby constructing a second oral model, and comparing the first oral model and the second oral model to obtain the soft tissue difference area between the two, and mark it in the second oral model.

[0134] The terminal device may be a computing device such as a desktop computer, laptop, PDA, or cloud server. The terminal device may include, but is not limited to, a processor and memory. Those skilled in the art will appreciate that the schematic diagram is merely an example of a terminal device and does not limit the terminal device. The terminal device may include more or fewer components than shown, or a combination of certain components, or different components. For example, the terminal device may also include input / output devices, network access devices, buses, and the like.

[0135] The processor may be a central processing unit (CPU), other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor, etc. The processor is the control center of the terminal device, connecting various parts of the entire terminal device using various interfaces and lines.

[0136] The memory can be used to store the computer programs and / or modules. The processor implements various functions of the terminal device by running or executing the computer programs and / or modules stored in the memory and calling the data stored in the memory. The memory can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system, at least one application required for a function, etc.; the data storage area can store data created based on the use of the mobile terminal, etc. In addition, the memory can include a high-speed random access memory and can also include a non-volatile memory, such as a hard disk, a memory, a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, at least one disk storage device, a flash memory device, or other volatile solid-state storage device.

[0137] Wherein, if the module / unit integrated in the terminal device is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present invention implements all or part of the process in the above-mentioned embodiment method, and can also be completed by a computer program to instruct the relevant hardware. The computer program can be stored in a computer-readable storage medium. When the computer program is executed by a processor, it can implement the steps of each of the above-mentioned method embodiments. Wherein, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording medium, USB flash drive, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electric carrier signal, telecommunication signal and software distribution medium. It should be noted that the content contained in the computer-readable medium can be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer-readable media do not include electric carrier signals and telecommunication signals.

[0138] Embodiment 4. Accordingly, the present invention also provides a computer-readable storage medium, which includes a stored computer program, wherein when the computer program is running, the device where the computer-readable storage medium is located is controlled to execute the image generation method based on oral imaging as described in any one of the above embodiments.

[0139] The specific embodiments described above further illustrate the objectives, technical solutions, and beneficial effects of the present invention. It should be understood that the above descriptions are merely specific embodiments of the present invention and are not intended to limit the scope of protection of the present invention. In particular, it should be noted that any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included within the scope of protection of the present invention for those skilled in the art.

Claims

1. An image generation method based on oral imaging, characterized in that: include: Collecting CT image data of the oral cavity to be processed, and constructing an oral bone model based on the CT image data; Extracting the bone features of the oral bone model, and covering the oral bone model with soft tissue according to the bone features through a preset healthy soft and hard tissue oral model database, to obtain a first oral model; wherein the preset healthy soft and hard tissue oral model database stores an oral model containing healthy soft and hard tissues; Performing a three-dimensional laser scan on the oral cavity to be treated, thereby constructing a second oral cavity model, and comparing the first oral cavity model with the second oral cavity model to obtain soft tissue difference areas between the two, and marking the areas in the second oral cavity model; Collecting optical image information of a target acquisition area in the oral cavity to be processed, and performing soft tissue feature extraction on the optical image information; wherein the target acquisition area corresponds to the soft tissue difference area marked on the second oral cavity model; The extracted soft tissue features are overlaid on the second oral cavity model to obtain a three-dimensional image model of the oral cavity to be processed, and then the oral cavity image is collected from the three-dimensional image model to generate an oral cavity image.

2. The method for generating an image based on oral imaging according to claim 1, characterized in that: The step of collecting CT image data of the oral cavity to be processed and constructing an oral bone model according to the CT image data is specifically as follows: Through the cone beam imaging device, the oral cavity to be treated is scanned layer by layer according to the preset scanning interval to obtain the CT image data corresponding to each layer; Perform bone hard tissue recognition on the CT image data corresponding to each layer, and extract hard tissue feature model data corresponding to each CT image data; sequentially correcting the hard tissue feature model data extracted from the CT image data corresponding to each layer, so that in each process of correcting the hard tissue feature model data of one layer, the hard tissue feature model data adjacent to the layer is used as the corrected reference model data, and the corrected reference model data is combined with the hard tissue feature model data of the layer, so as to calculate the first edge smoothness between the combined corrected reference model data and the hard tissue feature model data of the layer, and then correct the hard tissue feature model data of the layer according to the first edge smoothness; The corrected hard tissue feature model data of each layer are combined and merged to construct an oral bone model.

3. The method for generating an image based on oral imaging according to claim 2, characterized in that: The step of correcting the hard tissue feature model data of the layer according to the first edge smoothness is specifically as follows: When the first edge smoothness is within a preset threshold range, there is no need to correct the hard tissue feature model data of the layer; When the first edge smoothness is outside the preset threshold range, the second edge smoothness between the hard tissue feature model data of each layer in the corrected reference model data is calculated, and when the difference between the first edge smoothness and the second edge smoothness is greater than a preset error, the edge of the hard tissue feature model data of the layer is smoothed and corrected according to the second edge smoothness, thereby completing the correction of the hard tissue feature model data of the layer; Until the hard tissue feature model data of each layer is corrected.

4. The method for generating an image based on oral imaging according to claim 3, characterized in that: The step of extracting the bone features of the oral bone model and covering the oral bone model with soft tissue according to the bone features through a preset healthy soft and hard tissue oral model database to obtain a first oral model is specifically as follows: Extract the bone features of the preset key areas in the oral bone model; wherein the preset key areas include the upper the maxillary region, the mandibular region, the maxillary alveolar region, and the mandibular alveolar region; In a preset healthy soft and hard tissue oral cavity model database, a soft and hard tissue oral cavity model with the greatest similarity to the bone features of the preset key area is retrieved as a reference model; wherein the healthy soft and hard tissue oral cavity model database pre-stores several soft and hard tissue oral cavity models corresponding to the bone features of different preset key areas; Soft tissue data of the reference model is extracted, and the soft tissue data is overlaid on the oral skeletal model to construct a first oral model.

5. The method for generating an image based on oral imaging according to claim 4, characterized in that: The oral cavity to be treated is subjected to a three-dimensional laser scan to construct a second oral cavity model, and the first oral cavity model and the second oral cavity model are compared to obtain the soft tissue difference areas between the two, and the areas are marked in the second oral cavity model, specifically: Performing a three-dimensional laser scan on the oral cavity to be treated to obtain an oral cavity structure model; The oral bone model is matched with the oral structure model to construct a second oral model; Comparing the first oral cavity model with the second oral cavity model, and sequentially obtaining regions where there are difference structures between the first oral cavity model and the second oral cavity model; If the region of the difference structure exists in the first oral cavity model, the region of the difference structure is eliminated; If the region of the difference structure exists in the second oral cavity model, the region of the difference structure is taken as a soft tissue difference region and marked in the second oral cavity model.

6. The method for generating an image based on oral imaging according to claim 5, characterized in that: The collecting of optical image information of the target collection area in the oral cavity to be processed and the extraction of soft tissue features from the optical image information are specifically as follows: Collecting optical image information of a target acquisition area in the oral cavity to be processed, and sequentially performing pathological identification on the optical image information according to the pathological identification model; wherein the target acquisition area corresponds to the soft tissue difference area marked on the second oral cavity model; The soft tissue features are identified from the optical image information with the identified pathological features, so as to extract the soft tissue feature data with the pathological features.

7. The method for generating an image based on oral imaging according to claim 6, characterized in that: The step of overlaying the extracted soft tissue features onto the second oral cavity model to obtain a three-dimensional image model of the oral cavity to be processed is specifically as follows: According to the target acquisition area of ​​the optical image information of the pathological feature identified, the soft tissue feature data with the pathological feature extracted are sequentially covered in the second oral model of the corresponding area; The soft tissue difference regions corresponding to the optical image information of the second oral cavity model where no physical features are identified are marked and removed to obtain the final second oral cavity model; According to the soft tissue data in the first oral cavity model, the unlabeled area in the final second oral cavity model is covered with soft tissue, so as to obtain a three-dimensional image model of the oral cavity to be processed.

8. An image generation system based on oral imaging, characterized in that: include: Acquisition equipment and processing host; The acquisition equipment includes: cone beam imaging equipment, laser three-dimensional scanning equipment and optical image acquisition equipment; The processing host comprises: a CT image module, a first oral cavity model module, a second oral cavity model module, a soft tissue feature module and a soft tissue coverage module; The CT image module is used to collect CT image data of the oral cavity to be processed, and to construct an oral bone model according to the CT image data; The first oral model module is used to extract the bone features of the oral bone model and A soft and hard tissue oral model database, covering the oral skeletal model with soft tissue according to the skeletal features to obtain a first oral model; wherein the preset healthy soft and hard tissue oral model database stores an oral model containing healthy soft and hard tissues; The second oral cavity model module is used to perform a three-dimensional laser scan of the oral cavity to be processed, so as to construct a second oral cavity model, and compare the first oral cavity model with the second oral cavity model to obtain the soft tissue difference area between the two, and mark it in the second oral cavity model; The soft tissue feature module is used to collect optical image information of a target collection area in the oral cavity to be processed, and to extract soft tissue features from the optical image information; wherein the target collection area corresponds to the soft tissue difference area marked on the second oral cavity model; The soft tissue covering module is used to cover the extracted soft tissue features into the second oral cavity model to obtain a three-dimensional image model of the oral cavity to be processed, and then collect an oral cavity image from the three-dimensional image model to generate an oral cavity image.

9. A terminal device, characterized in that: The method comprises a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, wherein the processor implements the image generation method based on oral imaging according to any one of claims 1 to 7 when executing the computer program.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium includes a stored computer program; wherein, when the computer program is run, it controls the device where the computer-readable storage medium is located to execute the image generation method based on oral imaging according to any one of claims 1 to 7.

Images