Color fusion method and device, electronic equipment, storage medium and program product

Abstract

The embodiment of the invention provides a color fusion method and device, electronic equipment, a storage medium and a program product, and relates to the technical field of oral medicine. According to the method, point clouds of different visual angles are used for registration to obtain a point cloud model, and meshing processing is performed on the point cloud model to obtain a tooth model. And fusing the original RGB of the plurality of neighborhood points corresponding to each point in the point cloud model, and updating the RGB value of each point in the point cloud model into the corresponding fused RGB. And searching a first vertex closest to each point in the tooth model in the point cloud model, and updating an RGB value corresponding to a second vertex in the tooth model by using the RGB value of the closest first vertex. According to the method, the color of the tooth model can be truly restored through color fusion of the plurality of neighborhood points, the oral cavity problem is more visually displayed to customers and doctors through the tooth model with the vivid color, and necessary preconditions are provided for subsequent tooth pathological diagnosis, so that the market recognition degree of the intraoral scanner is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of oral medicine, and in particular to a color fusion method, device, electronic equipment, storage medium and program product. Background Art

[0002] As people's living standards continue to improve, they pay more and more attention to oral problems. As a digital product that allows people to understand their dental conditions more directly, intraoral scanners are widely favored by users because of their easy operation and excellent user experience.

[0003] In the prior art, an intraoral scanner is used to scan teeth from different perspectives, and a tooth model is generated based on the scan data from different perspectives. Due to the presence of shadows, the color of the tooth model is not realistic enough and there are many color noise points, which will undoubtedly greatly reduce the market acceptance of intraoral scanners. Summary of the invention

[0004] In view of this, an object of the present invention is to provide a color fusion method, device, electronic device, storage medium and program product, which can truly restore the color of the tooth model, thereby improving the market recognition of intraoral scanners.

[0005] In order to achieve the above purpose, the technical solution adopted by the embodiment of the present invention is as follows:

[0006] In a first aspect, the present invention provides a color fusion method, the method comprising:

[0007] Generate a point cloud model and a tooth model according to point clouds from different perspectives; the point cloud model is obtained by splicing the registered point clouds; the tooth model is obtained by gridding the point cloud model;

[0008] Taking each point in the point cloud model as a first vertex, fusing the original RGB of each neighborhood point to obtain a fused RGB corresponding to the first vertex;

[0009] Each point in the tooth model is used as a second vertex, a first vertex closest to the second vertex is obtained in the point cloud model, and an RGB value of the second vertex is updated to a fused RGB value corresponding to the closest first vertex.

[0010] In an optional embodiment, generating a point cloud model and a tooth model according to point clouds from different perspectives includes:

[0011] The point clouds of different perspectives are registered to obtain the pose matrix corresponding to the point cloud of each perspective and the normal vector corresponding to each point in the point cloud;

[0012] Using the pose matrices to stitch the point clouds of each perspective to obtain the point cloud model;

[0013] Based on each of the normal vectors, the point cloud model is meshed to obtain the tooth model.

[0014] In an optional implementation, taking each point in the point cloud model as a first vertex and fusing the original RGB of each neighborhood point to obtain a fused RGB corresponding to the first vertex includes:

[0015] Performing a neighbor search on the first vertex according to a preset radius to obtain a plurality of neighborhood points;

[0016] Determining the inner product weight of the neighborhood point according to each of the neighborhood points, the pose matrix corresponding to the neighborhood point and the normal vector of the neighborhood point;

[0017] Determine the brightness weight of the neighborhood point according to the original RGB of each neighborhood point;

[0018] Determine the fused RGB corresponding to the first vertex according to the inner product weights, brightness weights, and original RGB of all neighborhood points corresponding to the first vertex.

[0019] In an optional implementation, the pose matrix further includes camera coordinates; and determining the inner product weight of the neighborhood point according to each of the neighborhood points, the pose matrix corresponding to the neighborhood point, and the normal vector of the neighborhood point includes:

[0020] According to each of the neighborhood points and the camera coordinates, a direction vector corresponding to the neighborhood point is obtained;

[0021] Obtaining the registered normal vector corresponding to each of the neighborhood points according to the pose matrix corresponding to each of the neighborhood points and the normal vector of the neighborhood point;

[0022] The inner product of the direction vector corresponding to each of the neighborhood points and the normal vector after registration is normalized to obtain the inner product weight of the neighborhood point.

[0023] In an optional implementation manner, determining the brightness weight of the neighborhood point according to the original RGB of each of the neighborhood points includes:

[0024] Obtaining the brightness value of the neighborhood point according to a preset ratio and the initial RGB of each of the neighborhood points;

[0025] The brightness weight of the neighborhood point is obtained according to each of the brightness values ​​and the brightness mapping table; the brightness mapping table is used to record the one-to-one correspondence between the brightness weight and the brightness value range.

[0026] In an optional implementation manner, determining the fused RGB corresponding to the first vertex according to the inner product weights, brightness weights, and original RGB of all neighborhood points corresponding to the first vertex includes:

[0027] The product of the inner product weight and the brightness weight of each of the neighborhood points is summed to obtain a weighted sum;

[0028] The inner product weight, the brightness weight, and the product of the original RGB of each of the neighborhood points are summed to obtain a color sum;

[0029] The ratio of the color sum to the weight sum is determined as the fused RGB corresponding to the first vertex.

[0030] In a second aspect, the present invention provides a color fusion device, the device comprising:

[0031] A processing module, used to generate a point cloud model and a tooth model according to point clouds of different viewing angles; the point cloud model is obtained by splicing the registered point clouds; the tooth model is obtained by gridding the point cloud model;

[0032] A fusion module, used to take each point in the point cloud model as a first vertex, fuse the original RGB of each neighborhood point, and obtain a fused RGB corresponding to the first vertex;

[0033] An updating module is used to take each point in the tooth model as a second vertex, obtain the first vertex closest to the second vertex in the point cloud model, and update the RGB value of the second vertex to the fused RGB corresponding to the nearest first vertex.

[0034] In a third aspect, the present invention provides an electronic device, comprising a processor and a memory, wherein the memory stores a computer program executable by the processor, and the processor can execute the computer program to implement the color fusion method described in any of the aforementioned embodiments.

[0035] In a fourth aspect, the present invention provides a computer-readable storage medium having a computer program stored thereon, wherein when the computer program is executed by a processor, the color fusion method as described in any one of the aforementioned embodiments is implemented.

[0036] In a fifth aspect, the present invention provides a program product, which, when executed by a processor, implements the color fusion method as described in any one of the aforementioned embodiments.

[0037] Compared with the prior art, the color fusion method, device, electronic device, storage medium and program product provided by the embodiments of the present invention use point clouds of different perspectives for registration to obtain a point cloud model, and mesh the point cloud model to obtain a tooth model. The original RGB of multiple neighborhood points corresponding to each point in the point cloud model is used for fusion, and the RGB value of each point in the point cloud model is updated to the corresponding fused RGB. The first vertex closest to each point in the tooth model is found in the point cloud model, and the RGB value corresponding to the second vertex in the tooth model is updated using the RGB value of the closest first vertex. Color fusion through multiple neighborhood points can truly restore the color of the tooth model, and oral problems can be more intuitively presented to customers and doctors through a color-realistic tooth model. It is simple to implement, efficient and accurate, and provides the necessary prerequisites for subsequent dental pathology diagnosis, thereby improving the market recognition of intraoral scanners.

[0038] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, preferred embodiments are given below and described in detail with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for use in the embodiments are briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present invention and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without creative work.

[0040] Figure 1 A schematic flow chart of a color fusion method provided by an embodiment of the present invention is shown.

[0041] Figure 2 A schematic diagram of a tooth model before color fusion provided by an embodiment of the present invention is shown.

[0042] Figure 3 A schematic diagram of a color-fused tooth model provided by an embodiment of the present invention is shown.

[0043] Figure 4 Another schematic flow chart of a color fusion method provided by an embodiment of the present invention is shown.

[0044] Figure 5 A block diagram of a color fusion device provided by an embodiment of the present invention is shown.

[0045] Figure 6 A block diagram of an electronic device provided by an embodiment of the present invention is shown.

[0046] Icon: 100 - electronic device; 110 - memory; 120 - processor; 130 - communication module; 300 - color fusion device; 301 - processing module; 302 - fusion module; 303 - update module. DETAILED DESCRIPTION

[0047] The following will be combined with the accompanying drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Generally, the components of the embodiments of the present invention described and shown in the drawings here can be arranged and designed in various different configurations.

[0048] Therefore, the following detailed description of the embodiments of the present invention provided in the accompanying drawings is not intended to limit the scope of the invention claimed for protection, but merely represents selected embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without making creative work are within the scope of protection of the present invention.

[0049] It should be noted that relational terms such as "first" and "second" are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0050] At present, a dental model is usually made by scanning the oral cavity with an intraoral scanner so that people can understand their dental conditions more intuitively. However, the color of the scanned dental model is determined by the image colors of multiple viewing angles.

[0051] The inventors have found that the colors of images taken from different perspectives will be different. If the colors of images from different perspectives are added together, there will be shadows, which will result in very bad colors of the tooth model presented in the end, which is not conducive to user experience, further diagnosis and treatment. Therefore, only by attaching accurate and realistic colors to the tooth model can doctors and users more accurately grasp the details of the diseased teeth, so as to better guide treatment, which is also a hot topic in the research of intraoral scanners.

[0052] The tooth models scanned by current scanners have problems such as insufficient color fidelity, low accuracy, high color noise, strong graininess, and poor user experience, which will undoubtedly greatly reduce the market acceptance of intraoral scanners. Therefore, how to use intraoral scanners to obtain a tooth model with accurate and realistic colors and enhance user acceptance of intraoral scanners is also urgent for oral digital products.

[0053] Based on this, the color fusion method and device provided in the embodiment of the present invention use point clouds of different perspectives to align to obtain a point cloud model, and mesh the point cloud model to obtain a tooth model. The original RGB of multiple neighborhood points corresponding to each point in the point cloud model is used for fusion, and the RGB value of each point in the point cloud model is updated to the corresponding fused RGB. The first vertex closest to each point in the tooth model is found in the point cloud model, and the RGB value corresponding to the second vertex in the tooth model is updated using the RGB value of the closest first vertex. Color fusion through multiple neighborhood points can truly restore the color of the tooth model, and oral problems can be more intuitively presented to customers and doctors through a color-realistic tooth model. It is simple to implement, efficient, and accurate, and provides the necessary prerequisites for subsequent dental pathology diagnosis, thereby improving the market recognition of intraoral scanners.

[0054] The embodiments of the present invention are described in detail below with reference to the accompanying drawings.

[0055] Please refer to Figure 1 , Figure 1 A schematic diagram of a color fusion method provided by an embodiment of the present invention is shown. The method comprises the following steps:

[0056] Step S10, generating a point cloud model and a tooth model according to point clouds of different viewing angles; the point cloud model is obtained by splicing the registered point clouds; and the tooth model is obtained by gridding the point cloud model.

[0057] In the embodiment of the present invention, an intraoral scanner is used to collect data in the oral cavity at different viewing angles to obtain point clouds at different viewing angles. The different viewing angles refer to different angles and positions. The point clouds at different viewing angles are aligned and then spliced ​​into a point cloud model, and then the point cloud model is meshed to obtain a tooth model, such as Figure 2 shown.

[0058] Step S20: taking each point in the point cloud model as the first vertex, fusing the original RGB of each neighborhood point to obtain the fused RGB corresponding to the first vertex.

[0059] In the embodiment of the present invention, the point cloud model is traversed, and the neighborhood points corresponding to each point in the point cloud model are searched for, and the original RGB of multiple neighborhood points corresponding to each point are fused to obtain the fused RGB corresponding to the point. After traversing the points in the point cloud model, the fused RGB corresponding to each point in the point cloud model is obtained, and each RGB value in the point cloud model is updated to the corresponding fused RGB.

[0060] Among them, Red Green Blue (RGB) is a model for representing colors. RGB divides colors into three independent parameters: red (R), green (G) and blue (B). Each color can have 256 different intensity levels (from 0 to 255), so 16.7 million colors can be combined.

[0061] Step S30, taking each point in the tooth model as a second vertex, obtaining the first vertex closest to the second vertex in the point cloud model, and updating the RGB value of the second vertex to the fused RGB corresponding to the nearest first vertex.

[0062] In the embodiment of the present invention, each vertex (i.e., the second vertex) in the tooth model is traversed, and for each second vertex, a nearest neighbor search algorithm such as a KD tree is used to search for the first vertex closest to the second vertex in the point cloud model, and the RGB value of the second vertex is updated to the fused RGB corresponding to the closest first vertex. The RGB values ​​corresponding to all points in the updated tooth model are fused RGB, such as Figure 3 shown.

[0063] In summary, the color fusion method provided in the embodiment of the present invention uses point clouds of different perspectives to align to obtain a point cloud model, and grids the point cloud model to obtain a tooth model. The original RGB of multiple neighborhood points corresponding to each point in the point cloud model is used for fusion, and the RGB value of each point in the point cloud model is updated to the corresponding fused RGB. The first vertex closest to each point in the tooth model is found in the point cloud model, and the RGB value corresponding to the second vertex in the tooth model is updated using the RGB value of the closest first vertex. Color fusion through multiple neighborhood points can truly restore the color of the tooth model, and oral problems can be more intuitively presented to customers and doctors through a color-realistic tooth model. It is simple to implement, efficient, and accurate, and provides the necessary prerequisites for subsequent dental pathology diagnosis, thereby improving the market recognition of intraoral scanners.

[0064] Optionally, a possible implementation method is provided below on how to use point clouds from different perspectives to generate a point cloud model and a tooth model. Figure 4 , Figure 1 The sub-steps of step S10 may include:

[0065] Step S101, registering point clouds of different perspectives to obtain a pose matrix corresponding to the point cloud of each perspective and a normal vector corresponding to each point in the point cloud.

[0066] Step S102, using each pose matrix to stitch the point clouds of each perspective to obtain a point cloud model.

[0067] In the embodiment of the present invention, due to the influence of various factors such as equipment accuracy, scanning angle, environmental interference, etc., there are position deviations between point clouds collected from different perspectives, and registration is required. The purpose of point cloud registration is to transform point clouds from multiple perspectives in space so that they can be accurately aligned to form a complete three-dimensional point cloud model.

[0068] Before point cloud registration, the normal vector of each point in the point cloud of different viewpoints is calculated. The iterative closest point algorithm can be used to register point clouds of different viewpoints. The iterative closest point algorithm iteratively finds the closest point pair between two point clouds and calculates an optimal transformation matrix (i.e., pose matrix) so that the source point in the closest point pair is transformed with the target point cloud in the closest point cloud pair using the pose matrix, and the aligned point clouds are spliced ​​to obtain a point cloud model.

[0069] It should be noted that the pose matrix may include rotation angles and translation distances. The point cloud is rotated and translated using the rotation angles and translation distances corresponding to the point clouds of each perspective, and the rotated and translated point clouds of different perspectives are spliced ​​to obtain a complete three-dimensional point cloud model.

[0070] Step S103: meshing the point cloud model based on the normal vectors to obtain a tooth model.

[0071] In the implementation of the present invention, the point cloud model can be meshed using a greedy projection triangulation algorithm to obtain a tooth model. Specifically, the corresponding pose matrix and normal vector are obtained according to the points in the point cloud model, the normal vector is transformed based on the pose matrix to obtain the aligned normal vector, a triangular face is generated according to the point cloud in the point cloud model and the aligned normal vector corresponding to the point cloud, a continuous surface is constructed, and each triangular face is connected to obtain a closed three-dimensional network model, i.e., a tooth model.

[0072] Optionally, a possible implementation method for generating fused RGB is provided below. Figure 4 , Figure 1 The sub-steps of step S20 may include:

[0073] Step S201 : performing a neighbor search on the first vertex according to a preset radius to obtain a plurality of neighborhood points.

[0074] In an embodiment of the present invention, the preset radius can be configured according to an empirical value, for example, the range radius containing about 10 points in the neighborhood is set to the preset radius. A neighbor search is performed on each first vertex in the point cloud model in turn to obtain multiple neighborhood points corresponding to each first vertex.

[0075] It should be noted that the first vertex and the neighboring points are relative. For example, for point A, points B and C are neighboring points. For point B, points A and D are neighboring points. A neighbor search algorithm such as a KD tree can be used to obtain multiple neighboring points of each first vertex.

[0076] Step S202, determining the inner product weight of the neighborhood point according to each neighborhood point, the pose matrix corresponding to the neighborhood point and the normal vector of the neighborhood point.

[0077] Step S203, determining the brightness weight of the neighborhood point according to the original RGB of each neighborhood point.

[0078] In the embodiment of the present invention, the pose matrix, normal vector and original RGB corresponding to each neighborhood point are obtained in turn, wherein the pose matrix is ​​the pose matrix corresponding to the point cloud where the neighborhood point is located. The inner product weight of each neighborhood point is calculated based on each neighborhood point, the pose matrix and the normal vector, and the brightness weight corresponding to the neighborhood point is determined based on the original RGB of each neighborhood point.

[0079] Step S204, determining the fused RGB corresponding to the first vertex according to the inner product weights, brightness weights and original RGB of all neighborhood points corresponding to the first vertex.

[0080] In an embodiment of the present invention, for each first vertex, after obtaining the inner product weight, brightness weight and original RGB of all neighborhood points corresponding to the first vertex, the fused RGB corresponding to the first vertex is determined using the full inner product weight, brightness weight and original RGB.

[0081] Optionally, the pose matrix also includes camera coordinates. As to how to obtain the inner product weight of each neighborhood point, a possible implementation method is provided below. Figure 4 The sub-steps of step S202 may include:

[0082] According to each neighborhood point and the camera coordinates, the direction vector corresponding to the neighborhood point is obtained; according to the pose matrix corresponding to each neighborhood point and the normal vector of the neighborhood point, the registered normal vector corresponding to the neighborhood point is obtained; the inner product of the direction vector corresponding to each neighborhood point and the registered normal vector is normalized to obtain the inner product weight of the neighborhood point.

[0083] In an embodiment of the present invention, for a first vertex, the pose matrix corresponding to the point cloud where each neighborhood point is located is obtained in turn, and the camera coordinates are obtained from the pose matrix. The direction vectors between the neighborhood points and the corresponding camera coordinates are calculated respectively to obtain the direction vectors corresponding to the neighborhood points. The normal vectors of the neighborhood points are adjusted using the pose matrix to obtain the registered normal vectors corresponding to the neighborhood points, the inner product of the direction vectors corresponding to the neighborhood points and the registered normal vectors is calculated, and the inner product is normalized to obtain the inner product weights of the neighborhood points.

[0084] The calculation formula of the inner product weight is as follows:

[0085] w k =normalize(V k ·P K )

[0086] Among them, w k is the inner product weight of the neighborhood point k; V k is the direction vector corresponding to the neighborhood point k; P k is the normal vector after registration corresponding to the neighborhood point k; normalize(·) is the normalization function.

[0087] Optionally, a possible implementation method is provided below for how to obtain the brightness weight of each neighborhood point. Figure 4 The sub-steps of step S203 may include:

[0088] According to the preset ratio and the initial RGB of each neighborhood point, the brightness value of the neighborhood point is obtained; according to each brightness value and the brightness mapping table, the brightness weight of the neighborhood point is obtained; the brightness mapping table is used to record the one-to-one correspondence between the brightness weight and the brightness value range.

[0089] In the embodiment of the present invention, a preset ratio is obtained, and the preset ratio includes a red ratio, a green ratio, and a blue ratio, wherein the sum of the red ratio, the green ratio, and the blue ratio is 1.

[0090] As a possible implementation, assume that the red ratio, green ratio and blue ratio in the preset ratio are 0.3334, 0.3333 and 0.3333 respectively. The initial RGB includes a red intensity level, a green intensity level and a blue intensity level.

[0091] The formula for calculating the brightness value is as follows:

[0092] Trgb=0.3334×R+0.3333×G+0.3333×B

[0093] Among them, Trgb is the brightness value of the neighborhood point; R is the red intensity level of the neighborhood point; G is the green intensity level of the neighborhood point; B is the blue intensity level of the neighborhood point.

[0094] According to the projection and reflection principle, when the brightness value is large, the probability of shadows will be low. Therefore, the brightness weights corresponding to different brightness value ranges can be set according to the brightness value. Assume that the brightness mapping table w trgb As shown below:

[0095]

[0096] According to the brightness value corresponding to the neighborhood point, the brightness mapping table is found, and the weight corresponding to the brightness value range in which the brightness value is located is determined as the brightness weight of the neighborhood point. For example, if the brightness value of the neighborhood point is 100, then the brightness weight corresponding to the neighborhood point is 0.15.

[0097] Optionally, a possible implementation method is provided below on how to determine the fused RGB using the inner product weight, brightness weight and original RGB of the neighborhood points. Figure 4 The sub-steps of step S204 may include:

[0098] The product of the inner product weight and the brightness weight of each neighborhood point is summed to obtain the weight sum; the product of the inner product weight, the brightness weight, and the original RGB of each neighborhood point is summed to obtain the color sum; the ratio of the color sum to the weight sum is determined as the fused RGB corresponding to the first vertex.

[0099] In the embodiment of the present invention, the calculation formulas of weight sum, color sum, and fusion RGB are as follows:

[0100]

[0101] P i =t RGB_i / t i

[0102] Among them, t i is the weight sum corresponding to the first vertex i; t RGB_i is the color and corresponding to the first vertex i; P i is the fused RGB corresponding to the first vertex i; w k is the inner product weight corresponding to the neighboring point k in the first vertex i; w trgb_k is the brightness weight corresponding to the neighboring point k in the first vertex i; RGB k is the initial RGB corresponding to the neighborhood point k in the first vertex i; n is the number of neighborhood points corresponding to the first vertex.

[0103] The above calculation formula is used to calculate the fused RGB corresponding to each point in the point cloud model, complete the color fusion of all points in the point cloud model, and obtain the point cloud model after fusion of RGB.

[0104] Based on the same inventive concept, the basic principle and technical effect of the color fusion device provided in the embodiment of the present invention are the same as those in the above embodiment. For the sake of brief description, for parts not mentioned in this embodiment, reference may be made to the corresponding contents in the above embodiment.

[0105] Please refer to Figure 5 , Figure 5 A block diagram of a color fusion device 300 provided in an embodiment of the present invention. The color fusion device 300 includes a processing module 301 , a fusion module 302 and an updating module 303 .

[0106] The processing module 301 is used to generate a point cloud model and a tooth model according to point clouds of different viewing angles; the point cloud model is obtained by splicing the registered point clouds; and the tooth model is obtained by gridding the point cloud model.

[0107] The fusion module 302 is used to take each point in the point cloud model as the first vertex, and fuse the original RGB of each neighborhood point to obtain the fused RGB corresponding to the first vertex.

[0108] The updating module 303 is used to take each point in the tooth model as a second vertex, obtain the first vertex closest to the second vertex in the point cloud model, and update the RGB value of the second vertex to the fused RGB corresponding to the nearest first vertex.

[0109] In summary, the color fusion device provided in the embodiment of the present invention uses point clouds of different perspectives to align to obtain a point cloud model, and grids the point cloud model to obtain a tooth model. The original RGB of multiple neighborhood points corresponding to each point in the point cloud model is used for fusion, and the RGB value of each point in the point cloud model is updated to the corresponding fused RGB. The first vertex closest to each point in the tooth model is found in the point cloud model, and the RGB value corresponding to the second vertex in the tooth model is updated using the RGB value of the closest first vertex. Color fusion through multiple neighborhood points can truly restore the color of the tooth model, and oral problems can be more intuitively presented to customers and doctors through a color-realistic tooth model. It is simple to implement, efficient, and accurate, and provides the necessary prerequisites for subsequent dental pathology diagnosis, thereby improving the market recognition of intraoral scanners.

[0110] Optionally, the processing module 301 is specifically used to align point clouds of different perspectives to obtain a pose matrix corresponding to the point cloud of each perspective and a normal vector corresponding to each point in the point cloud; use each pose matrix to splice the point clouds of each perspective to obtain a point cloud model; based on each normal vector, the point cloud model is gridded to obtain a tooth model.

[0111] Optionally, the fusion module 302 is specifically used to perform a neighbor search for the first vertex according to a preset radius to obtain multiple neighborhood points; determine the inner product weight of the neighborhood point according to each neighborhood point, the posture matrix corresponding to the neighborhood point and the normal vector of the neighborhood point; determine the brightness weight of the neighborhood point according to the original RGB of each neighborhood point; determine the fused RGB corresponding to the first vertex according to the inner product weight, brightness weight and original RGB of all neighborhood points corresponding to the first vertex.

[0112] Optionally, the pose matrix also includes camera coordinates; the fusion module 302 is specifically used to obtain the direction vector corresponding to the neighborhood point based on each neighborhood point and the camera coordinates; obtain the aligned normal vector corresponding to the neighborhood point based on the pose matrix corresponding to each neighborhood point and the normal vector of the neighborhood point; normalize the inner product of the direction vector corresponding to each neighborhood point and the aligned normal vector to obtain the inner product weight of the neighborhood point.

[0113] Optionally, the fusion module 302 is specifically used to obtain the brightness value of the neighborhood point according to a preset ratio and the initial RGB of each neighborhood point; obtain the brightness weight of the neighborhood point according to each brightness value and a brightness mapping table; the brightness mapping table is used to record the one-to-one correspondence between the brightness weight and the brightness value range.

[0114] Optionally, the fusion module 302 is specifically used to sum the product of the inner product weight and the brightness weight of each neighborhood point to obtain the weight sum; sum the product of the inner product weight, the brightness weight, and the original RGB of each neighborhood point to obtain the color sum; and determine the ratio of the color sum to the weight sum as the fused RGB corresponding to the first vertex.

[0115] Please refer to Figure 6 , is a block diagram of an electronic device 100 provided in an embodiment of the present invention. The electronic device 100 includes a memory 110, a processor 120, and a communication module 130. The memory 110, the processor 120, and the communication module 130 are electrically connected to each other directly or indirectly to achieve data transmission or interaction. For example, these components can be electrically connected to each other through one or more communication buses or signal lines.

[0116] The memory 110 is used to store programs or data. The memory 110 may be, but is not limited to, a random access memory (RAM), a read only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), etc.

[0117] The processor 120 is used to read / write data or programs stored in the memory 110 and execute corresponding functions. For example, when the computer program stored in the memory 110 is executed by the processor 120, the color fusion method disclosed in the above embodiments can be implemented.

[0118] The communication module 130 is used to establish a communication connection between the electronic device 100 and other communication terminals through a network, and to send and receive data through the network.

[0119] It should be understood that Figure 6 The structure shown is only a schematic diagram of the structure of the electronic device 100. The electronic device 100 may also include Figure 6 More or fewer components as shown, or with Figure 6 Different configurations are shown. Figure 6 Each component shown in the figure can be implemented by hardware, software or a combination thereof.

[0120] The embodiment of the present invention further provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by the processor 120, the color fusion method disclosed in the above embodiments is implemented.

[0121] The embodiment of the present invention further provides a program product, which implements the color fusion method disclosed in the above embodiments when executed by the processor 120.

[0122] In several embodiments provided in the present application, it should be understood that the disclosed devices and methods can also be implemented in other ways. The device embodiments described above are merely schematic. For example, the flowcharts and block diagrams in the accompanying drawings show the possible architecture, functions and operations of the devices, methods and computer program products according to multiple embodiments of the present invention. In this regard, each box in the flowchart or block diagram can represent a module, a program segment or a part of a code, and the module, a program segment or a part of a code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart, and the combination of boxes in the block diagram and / or flowchart can be implemented with a dedicated hardware-based system that performs a specified function or action, or can be implemented with a combination of dedicated hardware and computer instructions.

[0123] In addition, the functional modules in the various embodiments of the present invention may be integrated together to form an independent part, or each module may exist independently, or two or more modules may be integrated to form an independent part.

[0124] If the functions are implemented in the form of software function modules and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods described in each embodiment of the present invention. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0125] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A color fusion method, characterized in that: The method comprises: Generate a point cloud model and a tooth model according to point clouds from different perspectives; the point cloud model is obtained by splicing the registered point clouds; the tooth model is obtained by gridding the point cloud model; Taking each point in the point cloud model as a first vertex, fusing the original RGB of each neighborhood point to obtain a fused RGB corresponding to the first vertex; Each point in the tooth model is used as a second vertex, a first vertex closest to the second vertex is obtained in the point cloud model, and an RGB value of the second vertex is updated to a fused RGB value corresponding to the closest first vertex.

2. The color fusion method according to claim 1, characterized in that: The step of generating a point cloud model and a tooth model according to point clouds from different perspectives includes: The point clouds of different perspectives are registered to obtain the pose matrix corresponding to the point cloud of each perspective and the normal vector corresponding to each point in the point cloud; Using the pose matrices to stitch the point clouds of each perspective to obtain the point cloud model; Based on each of the normal vectors, the point cloud model is meshed to obtain the tooth model.

3. The color fusion method according to claim 2, characterized in that: The method of taking each point in the point cloud model as a first vertex and fusing the original RGB of each neighborhood point to obtain a fused RGB corresponding to the first vertex includes: Performing a neighbor search on the first vertex according to a preset radius to obtain a plurality of neighborhood points; Determining the inner product weight of the neighborhood point according to each of the neighborhood points, the pose matrix corresponding to the neighborhood point and the normal vector of the neighborhood point; Determine the brightness weight of the neighborhood point according to the original RGB of each neighborhood point; Determine the fused RGB corresponding to the first vertex according to the inner product weights, brightness weights, and original RGB of all neighborhood points corresponding to the first vertex.

4. The color fusion method according to claim 3, characterized in that: The pose matrix also includes camera coordinates; determining the inner product weight of the neighborhood point according to each of the neighborhood points, the pose matrix corresponding to the neighborhood point, and the normal vector of the neighborhood point, includes: According to each of the neighborhood points and the camera coordinates, a direction vector corresponding to the neighborhood point is obtained; Obtaining the registered normal vector corresponding to each of the neighborhood points according to the pose matrix corresponding to each of the neighborhood points and the normal vector of the neighborhood point; The inner product of the direction vector corresponding to each of the neighborhood points and the normal vector after registration is normalized to obtain the inner product weight of the neighborhood point.

5. The color fusion method according to claim 3, characterized in that: The determining the brightness weight of the neighborhood point according to the original RGB of each neighborhood point comprises: Obtaining the brightness value of the neighborhood point according to a preset ratio and the initial RGB of each of the neighborhood points; The brightness weight of the neighborhood point is obtained according to each of the brightness values ​​and the brightness mapping table; the brightness mapping table is used to record the one-to-one correspondence between the brightness weight and the brightness value range.

6. The color fusion method according to claim 3, characterized in that: The determining, according to the inner product weights, brightness weights, and original RGB of all neighborhood points corresponding to the first vertex, the fused RGB corresponding to the first vertex includes: The product of the inner product weight and the brightness weight of each of the neighborhood points is summed to obtain a weighted sum; The inner product weight, the brightness weight, and the product of the original RGB of each of the neighborhood points are summed to obtain a color sum; The ratio of the color sum to the weight sum is determined as the fused RGB corresponding to the first vertex.

7. A color fusion device, characterized in that: The device comprises: A processing module, used to generate a point cloud model and a tooth model according to point clouds of different viewing angles; the point cloud model is obtained by splicing the registered point clouds; the tooth model is obtained by gridding the point cloud model; A fusion module, used to take each point in the point cloud model as a first vertex, and fuse the original RGB of each neighborhood point to obtain a fused RGB corresponding to the first vertex; An updating module is used to take each point in the tooth model as a second vertex, obtain the first vertex closest to the second vertex in the point cloud model, and update the RGB value of the second vertex to the fused RGB corresponding to the nearest first vertex.

8. An electronic device, characterized in that: The invention comprises a processor and a memory, wherein the memory stores a computer program executable by the processor, and the processor can execute the computer program to implement the color fusion method according to any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the color fusion method according to any one of claims 1 to 6 is implemented.

10. A program product, characterized in that When the program product is executed by a processor, the color fusion method according to any one of claims 1 to 6 is implemented.

Images