Data display method and apparatus, device, and storage medium

Abstract

The present disclosure relates to a data display method and apparatus, a device, and a storage medium. The data display method comprises: acquiring a first captured image of a target object, wherein the first captured image is used for three-dimensional reconstruction of the target object; acquiring a second captured image of the target object, wherein the second captured image is used for identification of dental caries in the target object; on the basis of the first captured image and the second captured image, constructing a digital model of the target object, wherein the digital model comprises three-dimensional reconstruction data; and rendering and displaying the digital model of the target object. The method provided by the present disclosure improves the efficiency of clinical consultations while ensuring the accuracy of dental caries examination.

Description

Data display method, device, equipment and storage medium

[0001] This disclosure claims priority to a Chinese patent application filed with the Patent Office of China on March 21, 2024, with application number 202410325780.0 and invention name “A data display method, device, equipment and storage medium”, the entire contents of which are incorporated herein by reference. Technical Field

[0002] The present disclosure relates to the field of data display technology, and in particular to a data display method, apparatus, device, and storage medium. Background Art

[0003] Intraoral caries detection is a common oral examination method used to discover and evaluate the caries condition on teeth. Usually, common intraoral caries examination methods include visual inspection, X-ray inspection, fluorescent inspection, and impact equipment-assisted inspection, etc. However, the visual inspection method is easy to miss and cannot view the caries inside the teeth, X-ray inspection has certain damage to the human body, and imaging equipment cannot automatically identify caries.

[0004] With the development of 3D scanning technology, intraoral scanners are commonly used to collect 3D data of teeth and gums inside the mouth. Intraoral scanners can directly obtain 3D topographic data and color texture information from the oral cavity. Because the scan data can clearly and intuitively reflect the internal state of the oral cavity, it can provide data for dental restoration, dental implants, and the diagnosis and prevention of oral diseases. Therefore, there is an urgent need to provide a method for intraoral caries inspection based on 3D scanning data to improve the efficiency of medical treatment while ensuring the accuracy of the inspection. Summary of the Invention

[0005] In order to solve the above technical problems, the embodiments of the present disclosure provide a data display method, device, equipment and storage medium, which improve the efficiency of medical treatment while ensuring the accuracy of examination.

[0006] In a first aspect, an embodiment of the present disclosure provides a data display method, comprising:

[0007] Acquire a first acquired image of a target object, where the first acquired image is used for three-dimensional reconstruction of the target object;

[0008] Acquiring a second acquired image of the target object, where the second acquired image is used for caries identification of the target object;

[0009] constructing a digital model of the target object based on the first acquired image and the second acquired image, wherein the digital model includes three-dimensional reconstruction data;

[0010] The digital model of the target object is rendered and displayed.

[0011] Optionally, the second captured image is captured by an image capture device after near-infrared light and / or ultraviolet light is projected onto the target object and reflected by the target object.

[0012] Optionally, constructing a digital model of the target object according to the first acquired image and the second acquired image includes:

[0013] constructing the first three-dimensional reconstruction data according to the first acquired image;

[0014] The three-dimensional reconstruction data is associated with the second acquired image to construct the second three-dimensional reconstruction data.

[0015] Optionally, the digital model further includes caries identification data. After constructing the first three-dimensional reconstruction data based on the first acquired image, the method further includes:

[0016] Caries recognition is performed on the target object based on the first three-dimensional reconstruction data and the second acquired image using a pre-trained recognition model to generate the caries recognition data.

[0017] Optionally, the first captured image includes a reconstructed image, and the reconstructed image is captured by the image capture device after a structured light pattern is projected onto the target image and reflected by the target image.

[0018] Optionally, constructing the first three-dimensional reconstruction data according to the first acquired image includes:

[0019] The target object is three-dimensionally reconstructed based on the reconstructed image to obtain three-dimensional shape data, and the first three-dimensional reconstruction data includes the three-dimensional shape data.

[0020] Optionally, the first collected image further includes a texture image, and the texture image is acquired by the image collection device after white light is projected onto the target object and reflected by the target object;

[0021] The constructing the first three-dimensional reconstruction data according to the first acquired image includes:

[0022] The three-dimensional shape data is mapped based on the texture image to construct the first three-dimensional reconstruction data.

[0023] Optionally, the display interface displaying the digital model of the target object includes a first display area, and the rendering and displaying of the digital model of the target object includes:

[0024] Rendering and displaying target three-dimensional reconstruction data in the first display area;

[0025] Among them, the target three-dimensional reconstruction data refers to the first three-dimensional reconstruction data, the second three-dimensional reconstruction data and / or the third three-dimensional reconstruction data, the third three-dimensional data refers to the three-dimensional reconstruction data obtained by mapping the caries image in the second acquired image in a local area of ​​the first three-dimensional reconstruction data and mapping the texture image in the first acquired image in the remaining areas except the local area, and the local area refers to the caries area in the caries identification data.

[0026] Optionally, the display interface further includes a second display area and a target frame, and the second display area is used to display the second acquired image.

[0027] Optionally, after rendering and displaying the target three-dimensional reconstruction data in the first display area, the method further includes:

[0028] In response to the movement operation of the target frame on the target three-dimensional reconstruction data, a second acquired image corresponding to the target area is displayed in the second display area according to a preset ratio, wherein the target area refers to the area selected by the target frame on the target three-dimensional reconstruction data.

[0029] Optionally, the display interface further includes a third display area, which is used to display the texture image in the first acquired image and does not overlap with the second display area.

[0030] Optionally, in response to a movement operation of the target frame on the three-dimensional reconstruction data, the method further includes:

[0031] The texture image corresponding to the target area is displayed in the third display area according to the preset ratio, wherein the second display area and the third display area are displayed synchronously based on the movement operation.

[0032] Optionally, the caries identification data includes a caries position of each caries in the target object in the first three-dimensional reconstruction data and a caries sequence number of each caries.

[0033] Optionally, the rendering and displaying of the digital model of the target object includes:

[0034] displaying the first three-dimensional reconstruction data identifying the caries number and the caries position, and / or

[0035] Displaying the caries identification data in the form of text prompts, and / or,

[0036] During the scanning process of the target object, the caries identification data is broadcasted in real time by voice prompts.

[0037] In a second aspect, an embodiment of the present disclosure provides a data display device, comprising:

[0038] A first acquisition unit is configured to acquire a first acquired image of a target object, where the first acquired image is used for three-dimensional reconstruction of the target object;

[0039] a second acquiring unit, configured to acquire a second acquired image of the target object, wherein the second acquired image is used for caries identification of the target object;

[0040] a construction unit configured to construct a digital model of a target object according to the first acquired image and the second acquired image, wherein the digital model includes three-dimensional reconstruction data;

[0041] The display unit is configured to render and display the digital model of the target object.

[0042] In a third aspect, an embodiment of the present disclosure provides an electronic device, the electronic device comprising:

[0043] one or more processors;

[0044] a storage device configured to store one or more programs;

[0045] When the one or more programs are executed by the one or more processors, the one or more processors implement the data display method as described above.

[0046] In a fourth aspect, an embodiment of the present disclosure provides a computer-readable storage medium having a computer program stored thereon, which implements the data display method described above when executed by a processor.

[0047] The disclosed embodiment provides a data display method, including: acquiring a first captured image of a target object, the first captured image being used for three-dimensional reconstruction of the target object; acquiring a second captured image of the target object, the second captured image being used for caries identification of the target object; constructing a digital model of the target object based on the first captured image and the second captured image, the digital model including three-dimensional reconstruction data; and rendering and displaying the digital model of the target object. The disclosed method performs dental caries inspection using captured images acquired under different projection lights, thereby ensuring inspection accuracy. During or after the three-dimensional scanning process, the digital model constructed based on the images acquired under different conditions is displayed. By rendering and displaying the digital model, the user can promptly understand the status of dental diseases inside the oral cavity and can view it visually in real time, thereby preventing oral diseases and effectively improving medical treatment efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the present disclosure.

[0049] In order to more clearly illustrate the embodiments of the present disclosure or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0050] FIG1 is a flow chart of a data display method provided by an embodiment of the present disclosure;

[0051] FIG2 is a schematic diagram of a display interface provided by an embodiment of the present disclosure;

[0052] FIG3 is a schematic diagram of another display interface provided by an embodiment of the present disclosure;

[0053] FIG4 is a schematic diagram of another display interface provided by an embodiment of the present disclosure;

[0054] FIG5 is a schematic structural diagram of a data display device provided by an embodiment of the present disclosure;

[0055] FIG6 is a schematic structural diagram of an electronic device provided by an embodiment of the present disclosure. DETAILED DESCRIPTION

[0056] In order to more clearly understand the above-mentioned objectives, features and advantages of the present disclosure, the scheme of the present disclosure will be further described below. It should be noted that the embodiments of the present disclosure and the features therein can be combined with each other in the absence of conflict.

[0057] In the following description, many specific details are set forth to facilitate a full understanding of the present disclosure, but the present disclosure may also be implemented in other ways different from those described herein; it is obvious that the embodiments in the specification are only part of the embodiments of the present disclosure, rather than all of the embodiments.

[0058] Specifically, common methods for detecting intraoral caries include:

[0059] Visual inspection: The dentist uses the naked eye and oral examination instruments, such as a mirror and a probe, to directly observe the condition of the teeth and oral tissues, examining the color, shape, surface texture, and presence of cavities.

[0060] X-ray examination: X-ray is a common examination method that can show the internal structure and tissue of the teeth. X-ray images can help detect the location and extent of cavities within the teeth. This method is often used to detect caries hidden under the tooth surface.

[0061] Fluoroscopy: Fluoroscopy uses a special fluorescent dye or fluorescent camera to detect changes in the tooth surface and tiny cavities. The fluorescent dye adheres to the tooth and then illuminates it with a specific fluorescent light source, causing it to fluoresce in different colors. Cavities and lesions appear in different colors and brightness, helping the doctor determine the presence and extent of caries.

[0062] Detection instrument assistance: Modern technology also provides some computer-assisted caries detection instruments, such as laser scanners, infrared scanners, etc. These instruments can identify caries and lesions by scanning the tooth surface or tissue using specific physical principles and computational algorithms.

[0063] To address the above technical issues, the disclosed embodiments provide a data display method that acquires captured images of a target object when it reflects different light sources, constructs a three-dimensional model of the target object based on the captured images, and uses the different reflective properties of tooth caries areas to determine caries. Furthermore, a recognition model is used to classify and learn caries images, identify caries areas in the three-dimensional reconstructed data, and alert the user to the recognition results in real time, thereby rapidly preventing and diagnosing oral diseases. This is described in detail in one or more of the following embodiments.

[0064] Specifically, the data display method can be executed by a terminal or a server. Specifically, the terminal or the server constructs a digital model capable of identifying caries conditions based on the collected images acquired under different circumstances, and renders and displays the digital model in real time.

[0065] FIG1 is a flow chart of a data display method provided by an embodiment of the present disclosure, which is applied to a terminal, which can be understood as an electronic device such as a computer. The method specifically includes steps S101 to S104 as shown in FIG1 :

[0066] S101: Acquire a first acquired image of a target object, where the first acquired image is used for three-dimensional reconstruction of the target object.

[0067] The first captured image is an image reflected by the target object and captured by the image capture device when white light and structured light patterns are projected onto the target object.

[0068] It is understandable that the first acquired image can be obtained by scanning with a three-dimensional scanning device (hereinafter referred to as the scanning device). The three-dimensional scanning device can be an intraoral three-dimensional scanner. The scanning device includes a projection component, an illumination component, and an acquisition component. The projection component is configured to project a structured light pattern onto the target object. The illumination component is configured to project white light onto the target object. The projection component and the illumination component can be projected alternately periodically or simultaneously. The specific projection method is not limited. The acquisition component is configured to acquire the white light and structured light pattern reflected by the target object to obtain the first acquired image. The first acquired image generated by the acquisition component can be acquired in real time, a certain number of acquired images can be acquired, or all acquired images can be acquired directly. The first acquired image includes a reconstructed image and a color texture image.

[0069] S102: Acquire a second acquired image of the target object, where the second acquired image is used for caries identification of the target object.

[0070] The second captured image is captured by the image capture device after near-infrared light and / or ultraviolet light is projected onto the target object and reflected by the target object.

[0071] It is understandable that the second captured image can also be obtained by a scanning device. Specifically, the lighting component is configured to project near-infrared light and / or ultraviolet light onto the target object, and the collecting component is configured to collect near-infrared light and / or ultraviolet light reflected by the target object to obtain the second captured image, wherein the light source projected onto the target object in the lighting component includes at least one of white light, near-infrared light, and ultraviolet light. For example, the lighting component projects white light and near-infrared light onto the target object. White light and near-infrared light can be projected periodically and alternately, or simultaneously. The specific projection method is not limited and can be determined according to user needs. Near-infrared imaging is an imaging technology used to observe and obtain tissue structure and functional information. It uses a specific wavelength range of near-infrared spectroscopy (NIR) (usually between 600 and 1000 nanometers) for imaging.

[0072] S103: Construct a digital model of the target object according to the first acquired image and the second acquired image.

[0073] The digital model includes three-dimensional reconstruction data.

[0074] It is understood that, based on the above-described S101 and S102, first 3D reconstruction data of the target object is constructed in real time based on the acquired first captured image. The first 3D reconstruction data refers to a 3D model, and the target object may be an oral cavity. In one feasible application scenario, an intraoral scanner captures the user's oral cavity in real time to generate captured images and transmits them to a terminal. The terminal constructs a 3D model of the user's oral cavity based on the captured images in real time. The 3D model may specifically be a tooth model. The tooth model may be a model of complete teeth in the oral cavity, a model of a portion of teeth, a model of a single tooth, or a partial model of a tooth.

[0075] The digital model includes first three-dimensional reconstruction data and second three-dimensional reconstruction data.

[0076] Optionally, constructing a digital model of the target object according to the first acquired image and the second acquired image in S103 can be specifically implemented by the following steps:

[0077] First three-dimensional reconstruction data is constructed according to the first acquired image; and second three-dimensional reconstruction data is constructed by associating the first three-dimensional reconstruction data with the second acquired data.

[0078] As can be understood, a 3D reconstruction algorithm is used to perform 3D reconstruction based on the first acquired image to construct 3D reconstruction data of the target object, which is recorded as first 3D reconstruction data. The specific 3D reconstruction algorithm is not limited and can be determined according to user needs. The 3D reconstruction data can be understood as a 3D model. Subsequently, the second acquired image is associated with the first 3D reconstruction data to construct second 3D reconstruction data.

[0079] Optionally, associating the 3D reconstruction data with the second acquired image to construct second 3D reconstruction data can be specifically achieved through the following steps:

[0080] The second acquired image is positionally associated with the first three-dimensional reconstruction data to construct second three-dimensional reconstruction data.

[0081] It is understood that position association can be achieved by directly mapping the second acquired image onto the 3D reconstructed data, or by associating the second acquired image with a corresponding specific region in the first 3D reconstructed data. For example, the second acquired image and the first 3D reconstructed data can be positionally associated based on tooth positions, so that the second acquired image can be used to locate the specific tooth position in the first 3D reconstructed data. Alternatively, the second acquired image can be directly applied to the first 3D reconstructed data based on the tooth position.

[0082] Optionally, the digital model further includes caries identification data. After constructing the first three-dimensional reconstruction data according to the first acquired image, the method further includes:

[0083] The first 3D reconstruction data and the second captured image are used as inputs to a pre-built recognition model, and caries recognition is performed on the target object using the recognition model to generate caries recognition data. Alternatively, caries recognition is performed on the target object based on the first 3D reconstruction data and the second captured image using a pre-trained recognition model to generate caries recognition data.

[0084] For example, in one application scenario, a server trains a recognition model. A terminal obtains the trained recognition model from the server, uses the pre-trained recognition model to identify caries in a captured image, and displays the recognition result in real time. The captured image may be obtained by scanning with the terminal, which may be a scanning device. Alternatively, the captured image may be obtained by the terminal from another scanning device. Alternatively, the captured image may be obtained by the terminal performing image processing on a preset image, which may be scanned with another scanning device, or the preset image may be obtained by the terminal from another scanning device. The type of other scanning devices is not specifically limited herein.

[0085] In another application scenario, the server trains the recognition model. Furthermore, the server uses the pre-trained recognition model to identify caries in the acquired image and displays the recognition results in real time. The server acquires the acquired image in a manner similar to the terminal's acquisition of the acquired image, as described above, and will not be further described here.

[0086] In another application scenario, the terminal trains the recognition model. Further, the terminal uses the pre-trained recognition model to perform caries recognition on the acquired image and displays the recognition result in real time.

[0087] It can be understood that the first three-dimensional reconstruction data and the second acquired image are used as inputs of the recognition model, or the second three-dimensional reconstruction data with completed position association is used as input of the recognition model, and caries recognition is performed through the recognition model to identify the caries included in the target object and generate caries recognition data.

[0088] Among them, the first captured image also includes a reconstructed image and a texture image. The reconstructed image includes multiple images with different stripes, and the multiple images are obtained by capturing light in different preset bands; the reconstructed image is a structured light pattern projected onto the target image and captured by the image capture device after being reflected by it; the texture image is white light projected onto the target object and captured by the image capture device after being reflected by it.

[0089] It can be understood that the multiple images include a first stripe image and a second stripe image. The first stripe image is acquired by a black camera in the acquisition component, and the second stripe image is acquired by a color camera in the acquisition component. Different cameras acquire different stripe images under light of different bands. In one possible scenario, the first stripe image is acquired by acquiring light of the first band, and the second stripe image is acquired by simultaneously acquiring light of the second band and light of the third band. For example, the light of the first band emitted by the first light source in the projection component is recorded as the first light. The first band is 435-480, and the first light is blue light. The first stripe image acquired by the black camera is the blue stripe image. The second light source in the projection component emits light of two bands, the 605-600 band and the 500-560 band, that is, the 605-600 band is the second band, the second light is red light, the 500-560 band is the third band, and the third light is green light. The second stripe image acquired by the color camera is the red-green stripe image. Another possible scenario is that the multiple images also include a third fringe image. In this case, each image captures light of a band respectively, that is, the first fringe image is obtained by capturing light of a first band, the second fringe image is obtained by capturing light of a second band, and the third fringe image is obtained by capturing light of a third band. The first band, the second band, and the third band are different. It is understandable that the color camera will also capture the color texture image of the target object.

[0090] It is understandable that the following embodiments are described by taking as an example the case where the first stripe image is a blue stripe image and the second stripe image is a red and green stripe image.

[0091] Optionally, constructing the first three-dimensional reconstruction data according to the first acquired image can be specifically implemented through the following steps:

[0092] Based on the reconstruction, the target object is three-dimensionally reconstructed to obtain three-dimensional shape data, and the first three-dimensional reconstruction data includes the three-dimensional shape data; the three-dimensional shape data is mapped based on the texture image to construct the first three-dimensional reconstruction data.

[0093] Optionally, performing three-dimensional reconstruction of the target object based on the reconstructed image to obtain three-dimensional shape data includes:

[0094] The second fringe image is used as a reconstruction image to perform three-dimensional reconstruction of the target object; the second fringe image is used as a coding image to determine the coding order corresponding to each stripe; the coding sequence of each stripe is determined based on the coding image, and stripe matching is performed on the stripes in the reconstructed image based on the coding sequence to determine the matching relationship; based on the matching relationship, a fringe reconstruction algorithm is used to perform three-dimensional reconstruction according to the reconstructed image, and a splicing and fusion algorithm is used to construct three-dimensional morphological data of the target object based on the reconstruction.

[0095] It can be understood that the red and green stripe image is used as a coding image, and the combination of red (0, 1) and green (0, 1) is used as coding information. Based on the coding information, the position of each stripe can be known, that is, the order of the red, green and blue stripes during projection is known, and the blue stripe can be determined through the red and green stripes. The red and green stripe image is used as a coding image to determine the order of each stripe, and the blue stripe image is used as a reconstruction image to obtain the three-dimensional coordinates of the object. Specifically, the light of different bands obtained by different cameras is divided into two colors (red and green). A preset encoding composed of two colors is used to identify and match the sequence of blue stripes. The stripe 3D reconstruction algorithm is then used to reconstruct based on the blue stripe image. The splicing and fusion algorithm is used to construct the three-dimensional shape data of the object. Subsequently, the color inside the mouth is determined based on the color texture map to facilitate rendering of teeth and gums, etc., to obtain a tooth model.

[0096] It is understood that, based on the above-described S101 and S102, the first 3D reconstructed data and the second captured image are used as inputs to a pre-established recognition model, which then performs caries identification on the target object, i.e., identifies the presence of caries in the oral cavity, and generates caries identification data, which refers to the identification result. The recognition model is trained using deep learning on a large number of caries images labeled with caries. The specific network structure and training method of the recognition model are not limited herein. The following embodiment is described in detail using the second collected image including a near-infrared image as an example. During the recognition process, the recognition model not only considers the presentation of tooth surface caries in the three-dimensional model, but also considers the flocculent pattern of the caries image presented in the near-infrared image obtained based on the principle of near-infrared imaging and the near-infrared absorption characteristics. It can be understood that the principle of near-infrared imaging in caries detection refers to the absorption characteristics of near-infrared light by tooth tissue and the changes in tissue structure caused by caries lesions in the teeth. The near-infrared absorption characteristics refer to the fact that near-infrared light has a large penetration depth in tooth tissue, but enamel and dentin have low absorption of near-infrared light. Lesions or caries lesions in the enamel can cause changes in tissue structure and optical properties, resulting in increased absorption of near-infrared light. In other words, the location where caries appear in the near-infrared image is brighter than the location without lesions. Enamel is the hardest part of human bone, wrapping around the surface of the tooth crown and appearing milky white. Therefore, the recognition model can identify internal tooth caries based on the absorption characteristics of near-infrared light presented in the near-infrared image, and can also identify surface tooth caries based on texture characteristics, resulting in more accurate and comprehensive recognition results.

[0097] S104: Render and display the digital model of the target object.

[0098] It can be understood that based on the above S101 to S103, the digital model of the selected target object is displayed through the display interface, wherein the display interface can be understood as a visualization interface, and the display interface includes a three-dimensional model, recognition results and collected images, that is, at least one of the three-dimensional model, recognition results and collected images can be displayed in the display interface.

[0099] In one embodiment, the display interface includes a first display area, which is used to display the target three-dimensional reconstruction data and render and display the digital model of the target object. This can be achieved by the following steps:

[0100] Render and display target three-dimensional reconstruction data in the first display area; wherein the target three-dimensional reconstruction data refers to the first three-dimensional reconstruction data, the second three-dimensional reconstruction data and / or the third three-dimensional reconstruction data, the third three-dimensional data refers to the three-dimensional reconstruction data obtained by mapping the caries image in the second acquired image in a local area of ​​the first three-dimensional reconstruction data and mapping the texture image in the first acquired image in the remaining area except the local area, and the local area refers to the caries area in the caries identification data.

[0101] It can be understood that the target three-dimensional reconstruction data is displayed in the first display area. The target three-dimensional reconstruction data can be a first three-dimensional reconstruction model with a texture image applied globally or locally, or a second three-dimensional reconstruction model with a near-infrared image applied globally or locally, or a third three-dimensional reconstruction model with a near-infrared image applied to the caries area and a texture image applied to the remaining areas except the caries area.

[0102] It can be understood that the caries area identified in the caries recognition data is determined, and the near-infrared image is applied to the caries area in the 3D reconstruction data. The texture image is applied to the non-caries area in the 3D reconstruction data, that is, the 3D reconstruction data after texture mapping is displayed. In one possible scenario, when a caries area is identified, the near-infrared image is applied to the caries area and the texture image is applied to the non-caries area to clarify the caries condition in the target object. Different mapping effects can also be displayed based on the caries recognition results after mapping, or the mapping can be directly based on the caries recognition results.

[0103] Optionally, the display interface further includes a second display area, and the second display area is used to display the second acquired image.

[0104] Optionally, the second display area is placed above the first display area.

[0105] For example, see Figure 2, which is a schematic diagram of a display interface provided by an embodiment of the present disclosure. The display interface includes a first display area and a second display area. The first display area can be understood as the main display area, which is used to display the target three-dimensional reconstruction data (tooth model). The second display area can be located above the first display area and is used to display the near-infrared image. Based on the three-dimensional model, the area where caries occurs on the tooth surface can be intuitively understood. At the same time, based on the brightness difference between the areas in the near-infrared image, the area where caries occurs inside the tooth can be intuitively understood. In a feasible embodiment, the display interface includes a first display area and a first identifier. In response to the triggering operation of the first identifier, the second display area is displayed on the first display area. The display position of the second display area in the first display area is not limited and can be placed above or below the first display area. The near-infrared image displayed in the second display area can be a complete tooth image after splicing and fusion, or it can be a partial tooth image. Alternatively, one or more near-infrared images to be displayed can be selected from multiple near-infrared images.

[0106] In one embodiment, the display interface further includes a target box.

[0107] Optionally, after rendering and displaying the target 3D reconstruction data in the first display area, the method further includes:

[0108] In response to the movement operation of the target frame on the target 3D reconstructed data, a second acquired image corresponding to the target area is displayed in the second display area according to a preset ratio, wherein the target area refers to the area selected by the target frame on the target 3D reconstructed data.

[0109] It is understandable that the display interface also includes a target box, which can be understood as a selection box. The target box can be moved on each display area. In one possible scenario, the target box moves in the first display area. Specifically, the target box can slide on the three-dimensional model. In response to the sliding operation of the target box on the three-dimensional model, the second acquired image corresponding to the target area is displayed in the second display area according to a preset ratio. The target area refers to the area selected by the target box on the three-dimensional model. For example, the area selected by the target box on the tooth model is the area where the incisors are located. The preset ratio can be set according to user needs. For example, a near-infrared image of the incisors can be displayed in equal proportions, that is, the second display area and the target box are displayed synchronously.

[0110] In one embodiment, the display interface further includes a third display area, which is used to display the texture image in the first acquired image. The third display area is placed above the first display area and does not overlap with the second display area.

[0111] Optionally, in response to a movement operation of the target frame on the target 3D reconstruction data, the method further includes:

[0112] The texture image corresponding to the target area is displayed in the third display area according to a preset ratio, wherein the second display area and the third display area are displayed synchronously based on the movement operation.

[0113] For example, see Figure 3, which is a schematic diagram of another display interface provided by an embodiment of the present disclosure. The display interface also includes a third display area, which is used to display a color texture image. In one possible scenario, the display interface includes a second identifier, a first display area, and a third display area. In response to the triggering operation of the second identifier, the third area is displayed. In another possible scenario, the display interface includes a first display area, a second display area, and a third display area. The second display area and the third display area are both located above the first display area, and the second display area and the third display area do not overlap. In this case, in response to the movement of the target frame on the three-dimensional model, the color texture image corresponding to the target area is displayed in the third display area according to a preset ratio. The target area also refers to the tooth area selected by the target frame. The area that can be selected by the target frame is not limited and can be set according to user needs. As shown in Figure 3, the second display area and the third display area display images synchronously in response to the movement of the target frame, that is, the second display area displays the color texture image of the target area, and the third display area displays the near-infrared image of the target area.

[0114] Optionally, the caries identification data includes the number of caries included in the target object, the caries position of each caries in the target object in the first three-dimensional reconstruction data, and the caries sequence number of each caries.

[0115] It can be understood that the recognition results include the number of caries included in the target object, the position of each caries in the three-dimensional model and the serial number of each caries. The number of caries can be understood as the number of teeth with caries, that is, a single tooth is used as the statistical unit, or the number of caries areas in the mouth, that is, the area with caries is used as the statistical unit. Multiple caries areas may appear on a single tooth. The caries position refers to the position of the caries area in the three-dimensional model. The caries serial number is determined based on the order of the caries.

[0116] Optionally, a digital model of the target object is rendered and displayed, including:

[0117] Display the first three-dimensional reconstruction data that identifies the caries number and caries location, and / or display the caries identification data through text prompts, and / or broadcast the caries identification data in real time through voice prompts during the scanning of the target object.

[0118] It is understandable that the recognition results can be directly marked on the three-dimensional model, for example, the serial number and specific area can be marked on each caries area, etc., and the caries recognition status and the number of caries can also be displayed in the form of text prompts on the display interface. The text can be marked on the three-dimensional model or directly displayed in a local position on the display interface. It is also possible to use a sticking method to stick the near-infrared image on the three-dimensional model, for example, stick the near-infrared image directly on the three-dimensional model, and for example, only stick the area where caries are identified to highlight it. It is also possible to generate a recognition result in the process of scanning the target object, and then broadcast the caries situation in real time through voice prompts, and accumulate the number of caries detected, and present the number of caries identified in real time in a numerical form.

[0119] For example, see FIG4 , which is a schematic diagram of another display interface provided by an embodiment of the present disclosure. The display interface shown in FIG4 identifies the serial number of each caries and selects each caries area according to the caries, so that the user can intuitively understand the caries situation. For example, the target object includes 3 caries areas, and the caries serial numbers of the 3 caries areas are caries 1, caries 2 and caries 3, and each caries area is marked at the same time. The caries area can be selected by the method shown in FIG4 , wherein caries 2 and caries 3 are tooth surface caries, and caries 1 is tooth internal caries. The number of caries can also be determined according to the caries serial number. At the same time, it can be seen from the near-infrared image displayed in the second display area that the brightness of the caries area is higher than that of other areas.

[0120] It is understandable that the display interface also includes a fourth display area, which is used to explain the morphology of caries with a visual animation. Users can understand the formation process and morphology of caries through the animation. At the same time, it can also be compared with the caries area that the user has identified, indicating that similar caries already exist in the user's mouth.

[0121] The disclosed embodiments provide a data display method that utilizes near-infrared imaging principles and a recognition model to identify caries within a target object. This method can identify caries not only on the tooth surface through texture, but also within the tooth itself, achieving high recognition accuracy and improving the accuracy and completeness of auxiliary diagnosis. The method can also alert users to caries within their mouths during or after the scan through voice prompts, text prompts, and 3D displays. This allows for real-time identification and prompting, effectively enhancing the scanning experience.

[0122] FIG5 is a schematic diagram of the structure of a data display device provided in an embodiment of the present disclosure. The data display device provided in an embodiment of the present disclosure can execute the processing flow provided in an embodiment of the data display method. As shown in FIG5 , the device 500 includes a first acquisition unit 501, a second acquisition unit 502, a construction unit 503, and a display unit 504, wherein:

[0123] A first acquisition unit 501 is configured to acquire a first acquired image of a target object, where the first acquired image is used for 3D reconstruction of the target object;

[0124] A second acquiring unit 502 is configured to acquire a second acquired image of the target object, where the second acquired image is used for caries identification of the target object;

[0125] A construction unit 503 is configured to construct a digital model of the target object according to the first acquired image and the second acquired image, wherein the digital model includes three-dimensional reconstruction data;

[0126] The display unit 504 is configured to render and display the digital model of the target object.

[0127] Optionally, the second captured image in the device 500 is captured by the image capture device after near-infrared light and / or ultraviolet light is projected onto the target object and reflected by the target object.

[0128] Optionally, the digital model in the device 500 includes first three-dimensional reconstruction data and second three-dimensional reconstruction data.

[0129] Optionally, the construction unit 503 is configured to:

[0130] constructing first three-dimensional reconstruction data according to the first acquired image;

[0131] The three-dimensional reconstruction data is associated with the second acquired image to construct second three-dimensional reconstruction data.

[0132] Optionally, the construction unit 503 is configured to:

[0133] Caries recognition is performed on the target object based on the first three-dimensional reconstruction data and the second acquired image using a pre-trained recognition model to generate caries recognition data.

[0134] Optionally, the first captured image in the device 500 includes a reconstructed image, which is acquired by the image capture device after the structured light pattern is projected onto the target image and reflected by the target image.

[0135] Optionally, the construction unit 503 is configured to:

[0136] The target object is three-dimensionally reconstructed based on the reconstructed image to obtain three-dimensional shape data, and the first three-dimensional reconstruction data includes the three-dimensional shape data.

[0137] Optionally, the first captured image in the device 500 further includes a texture image, and the texture image is captured by the image capture device after white light is projected onto the target object and reflected by the target object.

[0138] Optionally, the construction unit 503 is configured to:

[0139] The three-dimensional shape data is mapped based on the texture image to construct first three-dimensional reconstruction data.

[0140] Optionally, the display interface in the device 500 includes a first display area.

[0141] Optionally, the display unit 504 is configured to:

[0142] Rendering and displaying the target three-dimensional reconstruction data in the first display area;

[0143] Among them, the target three-dimensional reconstruction data refers to the first three-dimensional reconstruction data, the second three-dimensional reconstruction data and / or the third three-dimensional reconstruction data, the third three-dimensional data refers to the three-dimensional reconstruction data obtained by mapping the caries image in the second acquired image in a local area of ​​the first three-dimensional reconstruction data and mapping the texture image in the first acquired image in the remaining areas except the local area, and the local area refers to the caries area in the caries identification data.

[0144] Optionally, the display interface in the device 500 further includes a second display area and a target frame, and the second display area is used to display the second collected image.

[0145] Optionally, the display unit 504 is further configured to:

[0146] In response to a moving operation of the target frame on the target 3D reconstructed data, a second acquired image corresponding to the target area is displayed in the second display area according to a preset ratio, wherein the target area refers to the area selected by the target frame on the target 3D reconstructed data.

[0147] Optionally, the display interface in the device 500 further includes a third display area, which is used to display the texture image in the first acquired image and does not overlap with the second display area.

[0148] Optionally, the apparatus 500 is further configured to:

[0149] The texture image corresponding to the target area is displayed in the third display area according to a preset ratio, wherein the second display area and the third display area are displayed synchronously based on the movement operation.

[0150] Optionally, the caries identification data in the device 500 includes the caries position of each caries in the target object in the first three-dimensional reconstruction data and the caries sequence number of each caries.

[0151] Optionally, the apparatus 500 is further configured to:

[0152] Displaying 3D reconstruction data with caries number and location identified, and / or

[0153] Display caries identification data by means of text prompts, and / or,

[0154] During the scanning process of the target object, caries recognition data is reported in real time through voice prompts.

[0155] The data display device of the embodiment shown in FIG5 can be configured to execute the technical solution of the above-mentioned scanner embodiment. Its implementation principle and technical effects are similar and will not be described in detail here.

[0156] Figure 6 is a schematic diagram of the structure of an electronic device provided in an embodiment of the present disclosure. With specific reference to Figure 6 below, a schematic diagram of the structure of an electronic device 600 suitable for implementing the embodiment of the present disclosure is shown. The electronic device 600 in the embodiment of the present disclosure may include, but is not limited to, mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), wearable electronic devices, and the like, as well as fixed terminals such as digital TVs, desktop computers, smart home devices, and the like. The electronic device shown in Figure 6 is merely an example and should not impose any limitations on the functions and scope of use of the embodiments of the present disclosure.

[0157] As shown in Figure 6, electronic device 600 may include a processing scanner (e.g., a central processing unit, a graphics processing unit, etc.) 601, which can perform various appropriate actions and processes to implement the data display method of the embodiment as described in the present disclosure according to the program stored in the read-only memory (ROM) 602 or the program loaded from the storage scanner 608 into the random access memory (RAM) 603. In RAM 603, various programs and data required for the operation of electronic device 600 are also stored. Processing scanner 601, ROM 602, and RAM 603 are connected to each other via bus 604. Input / output (I / O) interface 605 is also connected to bus 604.

[0158] Typically, the following scanners may be connected to the I / O interface 605: an input scanner 606 including, for example, a touch screen, a touchpad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; an output scanner 607 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; a storage scanner 608 including, for example, a magnetic tape, a hard disk, etc.; and a communication scanner 609. The communication scanner 609 may allow the electronic device 600 to communicate with other devices wirelessly or by wire to exchange data. Although FIG. 6 illustrates an electronic device 600 with various scanners, it should be understood that not all of the illustrated scanners are required to be implemented or present. More or fewer scanners may alternatively be implemented or present.

[0159] In particular, according to an embodiment of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a non-transitory computer-readable medium, and the computer program includes a program code for executing the scanner shown in the flowchart, thereby implementing the data display method described above. In such an embodiment, the computer program can be downloaded and installed from the network via the communication scanner 609, or installed from the storage scanner 608, or installed from the ROM 602. When the computer program is executed by the processing scanner 601, the above-mentioned functions defined in the scanner of the embodiment of the present disclosure are performed.

[0160] It should be noted that the computer-readable medium mentioned above in the present disclosure may be a computer-readable signal medium or a computer-readable storage medium, or any combination of the two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, scanner, or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present disclosure, a computer-readable storage medium may be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, scanner, or device. In the present disclosure, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, which carries computer-readable program code. Such a propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium that can transmit, propagate, or convey a program for use by or in conjunction with an instruction execution system, scanner, or device. The program code contained on the computer-readable medium may be conveyed using any suitable medium, including but not limited to wires, optical cables, RF (radio frequency), or any suitable combination thereof.

[0161] In some embodiments, the client and server can communicate using any currently known or later developed network protocol, such as HTTP (HyperText Transfer Protocol), and can be interconnected with any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network ("LAN"), a wide area network ("WAN"), an internet (e.g., the Internet), and a peer-to-peer network (e.g., an ad hoc peer-to-peer network), as well as any currently known or later developed network.

[0162] The computer-readable medium may be included in the electronic device, or may exist independently without being incorporated into the electronic device.

[0163] Optionally, when the above one or more programs are executed by the electronic device, the electronic device may also execute other steps described in the above embodiments.

[0164] Computer program code for performing the operations of the present disclosure may be written in one or more programming languages, or a combination thereof, including, but not limited to, object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a stand-alone software package, partially on the user's computer and partially on a remote computer, or entirely on the remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., through the Internet using an Internet service provider).

[0165] The flow charts and block diagrams in the accompanying drawings illustrate the possible architecture, functions and operations of the systems, scanners and computer program products according to various embodiments of the present disclosure. In this regard, each box in the flow chart or block diagram can represent a module, program segment or a part of code, and the module, program segment or a part of code contains one or more executable instructions for realizing the prescribed logical functions. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a sequence different from that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flow chart, and the combination of the boxes in the block diagram and / or flow chart can be implemented by a dedicated hardware-based system that performs the prescribed function or operation, or can be implemented by a combination of dedicated hardware and computer instructions.

[0166] The units involved in the embodiments described in this disclosure may be implemented in software or hardware, wherein the name of a unit does not necessarily limit the unit itself.

[0167] The functions described above herein may be performed, at least in part, by one or more hardware logic components. For example, and without limitation, exemplary types of hardware logic components that may be used include: field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chip (SOCs), complex programmable logic devices (CPLDs), and the like.

[0168] In the context of the present disclosure, a machine-readable medium can be a tangible medium that can contain or store a program for use by or in conjunction with an instruction execution system, scanner, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, scanner, or device, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium can include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0169] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, scanner, article or gateway that includes a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements that are inherent to such process, scanner, article or gateway. In the absence of further restrictions, the elements defined by the sentence "including a..." do not exclude the presence of other identical elements in the process, scanner, article or gateway that includes the elements.

[0170] The foregoing description is intended only to provide specific embodiments of the present disclosure, intended to enable those skilled in the art to understand and implement the present disclosure. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present disclosure. Therefore, the present disclosure is not intended to be limited to the embodiments described herein, but rather to be construed in the broadest manner consistent with the principles and novel features disclosed herein. Industrial Applicability

[0171] The data display scheme provided by the embodiment of the present disclosure performs dental caries inspection by acquiring images under different projection lights, thereby ensuring the inspection accuracy. During the three-dimensional scanning process or after the scanning is completed, a digital model constructed based on the images acquired under different circumstances is displayed. By rendering and displaying the digital model, the user can promptly know the status of dental diseases inside the oral cavity and can view it visually in real time, thereby playing a role in preventing oral diseases, effectively improving the efficiency of medical treatment, and having strong industrial applicability.

Claims

1. A data display method, wherein: include: Acquire a first acquired image of a target object, where the first acquired image is used for three-dimensional reconstruction of the target object; Acquiring a second acquired image of the target object, where the second acquired image is used for caries identification of the target object; constructing a digital model of the target object based on the first acquired image and the second acquired image, wherein the digital model includes three-dimensional reconstruction data; The digital model of the target object is rendered and displayed.

2. The method according to claim 1, wherein The second collected image is obtained by an image collection device after near-infrared light and / or ultraviolet light is projected onto the target object and reflected by the target object.

3. The method according to claim 1, wherein The digital model includes first three-dimensional reconstruction data and second three-dimensional reconstruction data, and constructing the digital model of the target object according to the first acquired image and the second acquired image includes: constructing the first three-dimensional reconstruction data according to the first acquired image; The first three-dimensional reconstruction data is associated with the second acquired image to construct the second three-dimensional reconstruction data.

4. The method according to claim 3, wherein: The digital model also includes caries identification data. After constructing the first three-dimensional reconstruction data based on the first acquired image, the method further includes: Caries recognition is performed on the target object based on the first three-dimensional reconstruction data and the second acquired image using a pre-trained recognition model to generate the caries recognition data.

5. The method according to claim 3, wherein The first collected image includes a reconstructed image, and the reconstructed image is acquired by the image acquisition device after the structured light pattern is projected onto the target image and reflected by the target image; The constructing the first three-dimensional reconstruction data according to the first acquired image includes: The target object is three-dimensionally reconstructed based on the reconstructed image to obtain three-dimensional shape data, and the first three-dimensional reconstruction data includes the three-dimensional shape data.

6. The method according to claim 5, wherein: The first collected image further includes a texture image, wherein the texture image is acquired by the image collection device after white light is projected onto the target object and reflected by the target object; The constructing the first three-dimensional reconstruction data according to the first acquired image includes: The three-dimensional shape data is mapped based on the texture image to construct the first three-dimensional reconstruction data.

7. The method according to claim 4, wherein: The display interface displaying the digital model of the target object includes a first display area, The rendering displays the digital model of the target object, including: Rendering and displaying target three-dimensional reconstruction data in the first display area; Among them, the target three-dimensional reconstruction data refers to the first three-dimensional reconstruction data, the second three-dimensional reconstruction data and / or the third three-dimensional reconstruction data, the third three-dimensional data refers to the three-dimensional reconstruction data obtained by mapping the caries image in the second acquired image in a local area of ​​the first three-dimensional reconstruction data and mapping the texture image in the first acquired image in the remaining areas except the local area, and the local area refers to the caries area in the caries identification data.

8. The method according to claim 7, wherein: The display interface further includes a second display area and a target frame, the second display area being used to display the second acquired image. After rendering and displaying the target three-dimensional reconstruction data in the first display area, the method further includes: In response to the movement operation of the target frame on the target three-dimensional reconstruction data, a second acquired image corresponding to the target area is displayed in the second display area according to a preset ratio, wherein the target area refers to the area selected by the target frame on the target three-dimensional reconstruction data.

9. The method according to claim 8, wherein The display interface further includes a third display area, the third display area being used to display a texture image in the first acquired image and not overlapping with the second display area. In response to a movement operation of the target frame on the target three-dimensional reconstructed data, the method further includes: The texture image corresponding to the target area is displayed in the third display area according to the preset ratio, wherein the second display area and the third display area are displayed synchronously based on the movement operation.

10. The method according to claim 4, wherein: The caries identification data includes a caries position of each carious tooth in the target object in the first three-dimensional reconstruction data and a caries sequence number of each carious tooth. The rendering and displaying of the digital model of the target object includes: displaying the first three-dimensional reconstruction data identifying the caries number and the caries position, and / or Displaying the caries identification data in the form of text prompts, and / or, During the scanning process of the target object, the caries identification data is broadcasted in real time by voice prompts.

11. A data display device, wherein: include: A first acquisition unit is configured to acquire a first acquired image of a target object, where the first acquired image is used for three-dimensional reconstruction of the target object; a second acquiring unit, configured to acquire a second acquired image of the target object, wherein the second acquired image is used for caries identification of the target object; a construction unit configured to construct a digital model of a target object according to the first acquired image and the second acquired image, wherein the digital model includes three-dimensional reconstruction data; The display unit is configured to render and display the digital model of the target object.

12. An electronic device, wherein: The electronic device comprises: one or more processors; a storage device configured to store one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the data display method according to any one of claims 1 to 10.

13. A computer-readable storage medium having a computer program stored thereon, wherein: When the program is executed by a processor, the data display method according to any one of claims 1 to 10 is implemented.

Images