Data processing method and device, equipment and storage medium
By extracting and matching feature points in three-dimensional data, determining the coordinate transformation relationship, and combining semantic segmentation and optimization algorithms, the accuracy of three-dimensional data alignment at different time nodes is solved, and the accurate alignment and change display of three-dimensional data is achieved.
Patent Information
- Application Number
- CN202310521594.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2023-05-09
- Publication Date
- 2024-11-12
AI Technical Summary
在口腔诊疗等场景中,需要将不同时间节点下采集的三维数据在空间上对齐,以便比对患者部位的变化情况,但现有技术难以准确确定坐标变换关系,导致对齐结果不准确。
By obtaining the three-dimensional data of the same device source at different time nodes, extracting feature points and matching them, determining the target coordinate transformation relationship based on the matching feature points pairs, performing spatial alignment, and improving alignment accuracy through semantic segmentation and optimization algorithms.
The accurate alignment of three-dimensional data at different time nodes is achieved, the changes in the patient's location are shown, and the accuracy and reliability of alignment are improved.
Smart Images

Figure CN118941698A_ABST
Abstract
Description
Technical Field
[0001] The present disclosure relates to the field of data processing technology, and in particular to a data processing method, device, equipment and storage medium. Background Art
[0002] In some scenarios, it is necessary to determine the coordinate transformation relationship of the three-dimensional data collected by the same device source for the same target object at different time nodes, so as to spatially align the three-dimensional data collected at different time nodes. Taking the oral diagnosis and treatment scenario as an example, during the oral diagnosis and treatment process, three-dimensional data such as three-dimensional scanning data and CBCT data of the patient's oral cavity, face and other parts are usually collected. In order to facilitate the comparison of changes in various parts of the patient (for example, teeth) at different time nodes, the three-dimensional data collected at different time nodes need to be spatially aligned. Therefore, it is necessary to provide a solution that can accurately determine the coordinate transformation relationship of the three-dimensional data collected at different time nodes, so that the three-dimensional data collected at different time nodes can be displayed after being aligned in space. Summary of the invention
[0003] The present disclosure provides a data processing method, apparatus, device and storage medium.
[0004] According to a first aspect of an embodiment of the present disclosure, a data processing method is provided, the method comprising:
[0005] Acquire multiple sets of 3D data collected by the same device source for the target object at different time nodes;
[0006] For two sets of three-dimensional data at any two time nodes, feature points are extracted from the two sets of three-dimensional data respectively, and the extracted feature points are matched to obtain matching feature point pairs;
[0007] A target coordinate transformation relationship between the two sets of three-dimensional data is determined based on the feature point pairs, so as to spatially align the two sets of three-dimensional data based on the target coordinate transformation relationship.
[0008] According to a second aspect of an embodiment of the present disclosure, a data processing device is provided, the device comprising:
[0009] An acquisition module, used to acquire multiple sets of three-dimensional data collected by the same device source for the target object at different time nodes;
[0010] A feature point pair determination module is used to extract feature points from two sets of three-dimensional data at any two time nodes, respectively, and match the extracted feature points to obtain matching feature point pairs;
[0011] An alignment module is used to determine a target coordinate transformation relationship between the two sets of three-dimensional data based on the feature point pairs, so as to spatially align the two sets of three-dimensional data based on the target coordinate transformation relationship.
[0012] According to a third aspect of an embodiment of the present disclosure, an electronic device is provided, comprising a processor, a memory, and computer instructions stored in the memory and executable by the processor, wherein when the processor executes the computer instructions, the method mentioned in the first aspect above can be implemented.
[0013] According to a fourth aspect of an embodiment of the present disclosure, a computer-readable storage medium is provided, on which computer instructions are stored. When the computer instructions are executed, the method mentioned in the first aspect is implemented.
[0014] In the disclosed embodiment, after obtaining any two sets of three-dimensional data collected by the same device source at different time nodes for the same target object, feature points can be extracted from the two sets of three-dimensional data respectively, and the extracted feature points can be matched to obtain matching feature point pairs, and then the target coordinate transformation relationship of the two sets of three-dimensional data can be determined based on these matching feature point pairs, so that the two sets of three-dimensional data can be spatially aligned and displayed based on the target coordinate transformation relationship. In this way, the three-dimensional data collected for the same object at different time nodes can be aligned to display the changes in the three-dimensional data collected at different time nodes.
[0015] It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the disclosure. BRIEF DESCRIPTION OF THE DRAWINGS
[0016] The drawings herein are incorporated into the specification and constitute a part of the specification. These drawings illustrate embodiments consistent with the present disclosure and are used to illustrate the technical solutions of the present disclosure together with the specification.
[0017] Figure 1 It is a flow chart of data processing according to an embodiment of the present disclosure.
[0018] Figure 2 It is a schematic diagram of three-dimensional data alignment according to an embodiment of the present disclosure.
[0019] Figure 3 It is a schematic diagram of three-dimensional data alignment according to an embodiment of the present disclosure.
[0020] Figure 4 It is a schematic diagram of the logical structure of a data processing device according to an embodiment of the present disclosure.
[0021] Figure 5 It is a schematic diagram of the logical structure of a device according to an embodiment of the present disclosure. DETAILED DESCRIPTION
[0022] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present disclosure. Instead, they are merely examples of devices and methods consistent with some aspects of the present disclosure as detailed in the appended claims.
[0023] The terms used in this disclosure are only for the purpose of describing specific embodiments and are not intended to limit the disclosure. The singular forms of "a", "said" and "the" used in this disclosure and the appended claims are also intended to include plural forms, unless the context clearly indicates other meanings. It should also be understood that the term "and / or" used herein refers to and includes any or all possible combinations of one or more associated listed items. In addition, the term "at least one" herein means any combination of at least two of any one or more of a plurality of.
[0024] It should be understood that although the terms first, second, third, etc. may be used in the present disclosure to describe various information, such information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of the present disclosure, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information. Depending on the context, the word "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determining".
[0025] In order to enable those skilled in the art to better understand the technical solutions in the embodiments of the present disclosure and to make the above-mentioned purposes, features and advantages of the embodiments of the present disclosure more obvious and understandable, the technical solutions in the embodiments of the present disclosure are further described in detail below in conjunction with the accompanying drawings.
[0026] In some scenarios, it is necessary to determine the coordinate transformation relationship of the three-dimensional data collected by the same device source for the same target object at different time nodes, so as to spatially align the three-dimensional data collected at different time nodes. Taking the oral diagnosis and treatment scenario as an example, during the oral diagnosis and treatment process, three-dimensional data such as three-dimensional scanning data and CBCT data of the patient's oral cavity, face and other parts are usually collected. In order to facilitate the comparison of changes in various parts of the patient at different time nodes, the three-dimensional data collected at different time nodes need to be spatially aligned. For example, doctors usually need to compare the changes in patients' teeth at different time nodes, so they need to spatially align the three-dimensional data of teeth that are not collected through time nodes. Therefore, it is necessary to provide a solution that can accurately determine the coordinate transformation relationship of three-dimensional data collected at different time nodes, so that the three-dimensional data collected at different time nodes can be displayed after being aligned in space.
[0027] Based on this, the embodiment of the present disclosure provides a data processing method. After obtaining any two sets of three-dimensional data collected by the same device source for the same target object at different time nodes, feature points can be extracted from the two sets of three-dimensional data respectively, and the extracted feature points can be matched to obtain matched feature point pairs. Then, the target coordinate transformation relationship between the two sets of three-dimensional data can be determined based on these matched feature point pairs, so that any two sets of three-dimensional data can be spatially aligned and displayed based on the target coordinate transformation relationship.
[0028] The data processing method provided by the embodiments of the present disclosure can be implemented by a self-developed software code with data processing or viewing functions. For example, in some scenarios, the software code can be presented in the form of data viewing software, or can also be presented in the form of a web client with data viewing functions. Users can import the three-dimensional data collected by the same device source for the same target object at different time nodes into the data viewing software or web client, and the data viewing software or web client can align the imported three-dimensional data and display it.
[0029] The target object mentioned in the embodiments of the present disclosure may be various objects that require three-dimensional data collection and display. For example, the target object may be the patient's oral cavity, face, etc. The three-dimensional data may be the patient's oral cavity three-dimensional scanning data, facial three-dimensional scanning data, oral CBCT data, skull CBCT data, etc., which are not limited in the embodiments of the present disclosure.
[0030] Specifically, Figure 1 As shown, the data processing method provided by the embodiment of the present disclosure includes the following steps:
[0031] S102, obtaining multiple sets of three-dimensional data collected by the same device source for the target object at different time nodes;
[0032] In step S102, multiple sets of three-dimensional data collected by the same device source for the target object at different time nodes can be obtained. For example, taking the execution subject of the method as data viewing software, the user can import two or more sets of three-dimensional data collected by the same device source for the same target object at different time nodes in the data viewing software. Among them, the three-dimensional data can be three-dimensional point cloud data, three-dimensional mesh data, or volume data, etc., which is not limited by the embodiments of the present disclosure.
[0033] S104, for two sets of three-dimensional data at any two time nodes, extracting feature points from the two sets of three-dimensional data respectively, and matching the extracted feature points to obtain matching feature point pairs;
[0034] In step S104, for any two groups of three-dimensional data in the above-mentioned multiple groups of three-dimensional data, feature points can be extracted from the two groups of three-dimensional data respectively, wherein the feature points can be geometric feature points or texture feature points in the three-dimensional model represented by the three-dimensional data (for example, the three-dimensional model is a three-dimensional model carrying texture information), for example, the feature points can be some three-dimensional points with large geometric shape changes in the three-dimensional model, for example, they can be contour points; or some three-dimensional points with large texture changes. After extracting the respective feature points from the two groups of three-dimensional data, the feature points extracted from the two groups of three-dimensional data can be matched to obtain one or more groups of matched feature point pairs. Among them, the two feature points in the matched feature point pair correspond to the same three-dimensional point in the three-dimensional space, for example, the two feature points in the feature point pair both represent the center of the eyebrows, or the two feature points in the feature point pair both represent the protruding points of the zygomatic bones.
[0035] After extracting the feature points of the two sets of three-dimensional data, the matching feature point pairs may be determined based on the Ransca algorithm, or a similar algorithm may be used, which is not limited in the embodiments of the present disclosure.
[0036] S106: Determine a target coordinate transformation relationship between the two sets of three-dimensional data based on the feature point pairs, so as to spatially align the two sets of three-dimensional data based on the target coordinate transformation relationship.
[0037] In step S106, after determining the matched feature point pairs, a target coordinate transformation relationship representing the position transformation relationship of the two sets of three-dimensional data can be determined based on the matched feature point pairs, so as to align the two sets of three-dimensional data in space using the target coordinate transformation relationship. For example, if the two sets of three-dimensional data are two three-dimensional models, the rigid body transformation matrix of the two three-dimensional models can be calculated, which is the target coordinate transformation relationship.
[0038] Among them, for any two sets of three-dimensional data among the multiple sets of three-dimensional data, the target coordinate transformation relationship between the two sets of three-dimensional data can be determined in the above manner, so that the multiple sets of three-dimensional data can be finally aligned in space based on the target coordinate transformation relationship.
[0039] In some embodiments, in order to display the changes of three-dimensional data collected at different time nodes, after the multiple sets of three-dimensional data are spatially aligned based on the target coordinate transformation relationship between any two sets of three-dimensional data in the multiple sets of three-dimensional data, the changes of the multiple sets of three-dimensional data at different time nodes can be further displayed based on the alignment results.
[0040] In some embodiments, in order to better display the changes of the three-dimensional data collected by the same device source at different time nodes, the changes of the three-dimensional data collected by the same device source at different time nodes can be displayed in the form of animation. Since the acquisition process is discrete, the animation frames corresponding to the time points between two adjacent time nodes can be generated by interpolation animation. For example, continuous animation frames can be generated by Morphing technology to continuously present the changes of the three-dimensional data between adjacent time nodes.
[0041] Of course, in some scenarios, multiple sets of three-dimensional data collected at different time nodes can be superimposed together. Since the overlapping parts will overlap after superposition, the changes in non-overlapping areas (i.e., changes) can be highlighted.
[0042] As the target object changes over time, the shape of the target object will change to some extent. For example, in the oral cavity, the shape and arrangement of teeth will change to some extent over time. Therefore, it is difficult to establish an accurate correspondence between feature points. Therefore, the coordinate transformation relationship between the two sets of three-dimensional data determined based only on the matched feature points may not be accurate, resulting in inaccurate final alignment results. In order to obtain a more accurate coordinate transformation relationship, in some embodiments, such as Figure 2As shown, feature points (feature point 1, feature point 2) can be extracted from two sets of three-dimensional data (three-dimensional data 1, three-dimensional data 2) collected at different time nodes (for example, time node 1, time node 2), and then the feature points are matched to obtain feature point pairs, and the initial coordinate transformation relationship between the two sets of three-dimensional data is determined based on the feature point pairs. Through the initial coordinate transformation relationship, the two sets of three-dimensional data can be roughly aligned. After completing the rough alignment, the two sets of three-dimensional data can be semantically segmented to obtain two semantic entity sets (for example, semantic entity set 1, semantic entity set 2), wherein one semantic entity set corresponds to a set of three-dimensional data, that is, the semantic entities in the set are obtained by semantic segmentation of the set of three-dimensional data. Since the two sets of three-dimensional data both represent the same target object, there is a one-to-one correspondence between the semantic entities in the two semantic entity sets, that is, the two semantic entities representing the same object in the three-dimensional space in the two semantic entity sets can be associated. Furthermore, based on the correspondence between the semantic entities in the two semantic entity sets, the semantic entity pairs representing the same object can be determined, and based on these semantic entity pairs, the initial coordinate transformation relationship can be optimized to obtain a more accurate target coordinate transformation relationship, so as to use the target coordinate transformation relationship to precisely align the two sets of three-dimensional data. Compared with the correspondence between the feature points in the two sets of three-dimensional data, the correspondence between the semantic entities in the two sets of three-dimensional data is more accurate, and the correspondence between the semantic entities can be used to optimize the initial coordinate transformation relationship to obtain a more accurate coordinate transformation relationship.
[0043] In some embodiments, considering that the difference between two semantic entities representing the same object in three-dimensional space is usually small, therefore, when optimizing the initial coordinate transformation relationship based on the correspondence relationship between the semantic entities in the two semantic entity sets to obtain the target coordinate transformation relationship, one or more groups of target semantic entity pairs can be determined based on the correspondence relationship between the semantic entities in the two semantic entity sets, wherein the two semantic entities in the semantic entity pairs represent the same object in three-dimensional space. Then, the initial coordinate transformation relationship can be optimized based on the size of the difference between the two semantic entities in each group of target semantic entity pairs to obtain the target coordinate transformation relationship. For example, considering that the difference between two semantic entities representing the same object in three-dimensional space is generally small, therefore, when determining the target coordinate transformation relationship, the following optimization conditions can be set: solve the best target coordinate transformation relationship so that after the two semantic entities in each group of target semantic entity pairs are aligned in space using the target coordinate transformation relationship, the cumulative value of the difference between the two semantic entities in the one or more groups of target semantic entity pairs is as small as possible.
[0044] Of course, considering that different semantic entities may change over time, resulting in large differences between the two semantic entities in the same semantic entity pair. When optimizing the initial coordinate transformation relationship using semantic entity pairs, the optimization goal is usually to minimize the difference between the two semantic entities in the same semantic entity pair to obtain the best target coordinate transformation relationship. Therefore, if the initial coordinate transformation relationship is optimized using semantic entity pairs that change over time, since the two semantic entities in the semantic entity pair may have large differences, the target coordinate transformation relationship that is ultimately determined may not be accurate enough. Therefore, in some embodiments, such as Figure 3 As shown, in order to obtain a more accurate target coordinate transformation relationship, the target coordinate transformation relationship can be determined by using the semantic entity pairs composed of semantic entities that do not change with time. For example, it can be determined whether there are invariants in the two semantic entity sets, where the invariants are semantic entities whose morphology does not change with time. For example, taking the oral cavity as an example, some teeth in the oral cavity will not change with time, so these teeth can be used as invariants. Among them, the invariants can be pre-set by the user based on prior knowledge (such as oral treatment results, experience, etc.). If there are invariants in the two semantic entity sets, the invariants with corresponding relationships in the two semantic entity sets can be used to form a target semantic entity pair, and then the target semantic entity pair is used to optimize the initial coordinate transformation relationship.
[0045] By first determining the invariants from the semantic entities obtained by semantic segmentation of two sets of three-dimensional data, and then using the invariants with corresponding relationships to form a target semantic entity pair, since the two semantic entities of the target semantic entity pair have not changed, that is, the difference is basically 0, and then the difference between the two semantic entities in the target semantic entity pair is as small as possible as a constraint condition, the target coordinate transformation relationship is determined, thereby obtaining a more accurate target coordinate transformation relationship.
[0046] In some embodiments, the three-dimensional data collected at different time points may not have invariants. For example, taking the oral cavity as an example, as time changes, all teeth in the oral cavity have changes in geometric shape or arrangement posture. In this case, it is impossible to find invariants from the semantic entity sets of the two sets of three-dimensional data. Therefore, all semantic entity pairs with corresponding relationships in the two semantic entity sets can be used as target semantic entity pairs.
[0047] In some embodiments, if there is an invariant in the semantic entity set, that is, the target semantic entity pair is a semantic entity pair composed of invariants, in this case, the error is small, and the difference between the two semantic entities in the target semantic entity pair can be represented by the two-norm.
[0048] In some embodiments, if there is no invariant in the semantic entity set, that is, the target semantic entity pairs are all semantic entity pairs in the two semantic entity sets, since the two semantic entities in the target semantic entity pair have changed over time, there are differences between the two, which will cause errors. In order to concentrate the errors in a limited area, when characterizing the differences between the two semantic entities in the target semantic entity pair, a norm can be used to characterize them. For scenarios where there is no invariant in the semantic entity set, considering that there are many mismatched point pairs, a norm is used to characterize the differences to optimize the matching results, so that a more accurate coordinate transformation relationship can be obtained.
[0049] In some embodiments, the three-dimensional data may be a three-dimensional model of the oral cavity, and the semantic entity may be a single tooth model. The correspondence between the tooth models in the two three-dimensional models collected at different time points may be determined based on the tooth position identification (e.g., tooth position number) of the teeth. For example, for two sets of oral three-dimensional model data collected at different time points, semantic segmentation processing may be performed on them to obtain each tooth model and the tooth position number corresponding to the tooth model, and then based on the tooth position number of each tooth model in the two three-dimensional models, the correspondence between the teeth in the two three-dimensional models is established.
[0050] For example, take the alignment of the three-dimensional data collected from the oral cavity of the same patient at different time nodes as an example. Assume that the three-dimensional data records collected by the oral scanning device at different time nodes are in, is the maxillary data collected at time t (the superscript down represents the mandibular data), Represents the three-dimensional tooth model of the i-th tooth at time t. For each oral three-dimensional model at a certain time, semantic segmentation can be performed to obtain multiple tooth models, and then the tooth position number of each tooth model can be determined by tooth position recognition. The tooth models in the oral three-dimensional model at two time nodes can be associated through the tooth position number to establish a corresponding relationship between the tooth models.
[0051] The steps of aligning two sets of 3D data are divided into coarse alignment and fine alignment:
[0052] (1) Rough alignment: for t i ,t j Three-dimensional data of oral models collected at all times and Feature points are extracted separately to obtain geometric feature points, and the extracted set feature points are matched using the Ransca algorithm to obtain matching feature point pairs, and then the rigid body transformation matrix T of the two sets of three-dimensional data is calculated based on the feature point pairs. i,j .
[0053] (2) Fine alignment: Through the above rough alignment steps, and A preliminary alignment has been done within the same coordinate system.
[0054] (a) Since the teeth collected at different times may change (such as tooth morphology, tooth arrangement posture, tooth loss, etc.), it is impossible to establish a correspondence relationship in the entire domain (establish a correspondence relationship between three-dimensional points). Therefore, the treatment plan can be used to screen out t i ,t j A collection of teeth that remain unchanged at all times Among them, k represents the identifier of the tooth with unchanged shape. The tooth models in this set do not have changes in geometric shape and arrangement posture, so they can be considered as t i ,t j The key to aligning three-dimensional models is to find the accurate corresponding points between three-dimensional models. Therefore, the corresponding point pairs on the same tooth number in the tooth set are used to complete the precise alignment, that is, the best T can be determined by formula (1), and the final rigid body transformation matrix of the two sets of three-dimensional data is TTi,j:
[0055]
[0056] Where K represents the tooth model with unchanged morphology.
[0057] (b) Of course, in some scenarios, t i →t j All teeth have geometrical changes or arrangement postures at all times, so there is no actual invariant. The 1-norm energy can be used to measure the difference between tooth models, so that the error is concentrated in a limited area. For example, the optimal T can be determined by formula (2), and the final rigid body transformation matrix of the two sets of three-dimensional data is TTi,j
[0058]
[0059] Where K represents all tooth models.
[0060] In some embodiments, the three-dimensional data may be oral CBCT data. Since CBCT is volume data, before aligning two sets of CBCT data collected at different time points, isosurface extraction may be performed on the CBCT data to obtain surface mesh data. Feature points are then extracted from the surface mesh data, and the target coordinate transformation relationship is determined based on the feature point pairs.
[0061] In some embodiments, when determining the target coordinate transformation relationship between two sets of three-dimensional data based on feature point pairs, it is considered that over time, target objects such as the oral cavity and face tend to deform, and thus the obtained matching feature point pairs often have more mismatches. In this case, using the classical least squares method to obtain the target coordinate transformation relationship will not be very accurate. In order to obtain a more accurate target coordinate transformation relationship, when determining the target coordinate transformation relationship using feature point pairs, a norm can be used to characterize the difference between the two feature points in the feature point pair, and then the rigid body transformation matrix of the two sets of three-dimensional data is optimized based on the difference between the two feature points of the feature point pair to obtain the target coordinate transformation relationship. By using a norm to characterize the difference between feature point pairs, the final optimization result can be made more accurate and reliable.
[0062] In some embodiments, the feature point pair may include multiple groups. When determining the target coordinate transformation relationship of two groups of three-dimensional data based on the feature point pair, the target coordinate change relationship can be solved based on the constraint condition that the difference between the two feature points in the feature point pair is as small as possible. Considering that different types of feature points have different probabilities of change or different magnitudes of change over time. Therefore, when determining the target coordinate change relationship based on the difference between the two feature points in the feature point pair, the target coordinate transformation relationship can be determined based on the weighted average of the difference between the two feature points in each group of feature point pairs. Among them, considering that the probability of different feature points changing over time is different, different weights can be set for the differences of different feature point pairs. For example, for feature points that are likely to change over time, or feature points that will change greatly over time, the weight of their differences can be set smaller, and vice versa.
[0063] In some embodiments, the three-dimensional data is three-dimensional facial data, and the feature points may include one or more of the following: the center of the eyebrows, the prominent point of the cheekbones, the tip of the nose, the chin, etc.
[0064] For example, take the alignment of facial 3D data collected at different time points as an example. The facial 3D data collected by the facial scanning device at different time points is recorded as M t , feature points can be extracted from each face 3D data at different time nodes to obtain a set of feature points, such as eyes, nose tip, chin and other facial feature points The corresponding relationship between the feature points in the three-dimensional face data at different time nodes is established to obtain the matching feature point pairs. Then, the rigid body transformation matrix Ti,j of the two sets of three-dimensional data can be optimized based on the difference between the matching feature point pairs. Specifically, as shown in formula (3):
[0065]
[0066] Among them, ω k is the weight corresponding to each feature point, is the Kth feature point at time ti, is the Kth feature point at time tj.
[0067] Among them, considering that different types of feature points, such as eyes, nose tip, and chin, have different probabilities of changing over time, different weights can be set for the differences corresponding to different types of feature points. The weights of feature points with a higher probability of change can be smaller, and vice versa. In this way, the determined rigid body transformation matrix Ti,j is more accurate.
[0068] It is not difficult to understand that the solutions described in the above embodiments can be combined when there is no conflict, and they are not listed one by one in the embodiments of the present disclosure.
[0069] Accordingly, the present disclosure also provides a data processing device, such as Figure 4 As shown, the device 40 includes:
[0070] An acquisition module 41 is used to acquire multiple sets of three-dimensional data collected by the same device source for the target object at different time nodes;
[0071] A feature point pair determination module 42 is used to extract feature points from two sets of three-dimensional data at any two time nodes, respectively, and match the extracted feature points to obtain matching feature point pairs;
[0072] The alignment module 43 is used to determine the target coordinate transformation relationship between the two sets of three-dimensional data based on the feature point pairs, so as to spatially align the two sets of three-dimensional data based on the target coordinate transformation relationship.
[0073] Among them, the specific steps of the above-mentioned device executing the task processing method can refer to the description in the above-mentioned method embodiment, and will not be repeated here.
[0074] Furthermore, the present disclosure also provides a device, such as Figure 5 As shown, the device includes a processor 51, a memory 52, and computer instructions stored in the memory 52 for execution by the processor 51. When the processor 51 executes the computer instructions, the method described in any one of the above embodiments is implemented.
[0075] The present disclosure also provides a computer-readable storage medium on which a computer program is stored. When the program is executed by a processor, the method described in any of the above embodiments is implemented.
[0076] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.
[0077] It can be known from the description of the above implementation methods that those skilled in the art can clearly understand that the embodiments of the present disclosure can be implemented by means of software plus a necessary general hardware platform. Based on such an understanding, the technical solution of the embodiments of the present disclosure is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which can be stored in a storage medium such as ROM / RAM, a disk, an optical disk, etc., including several instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment of the present disclosure or some parts of the embodiments.
[0078] The systems, devices, modules or units described in the above embodiments may be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer, which may be in the form of a personal computer, a laptop computer, a cellular phone, a camera phone, a smart phone, a personal digital assistant, a media player, a navigation device, an email transceiver, a game console, a tablet computer, a wearable device or a combination of any of these devices.
[0079] Each embodiment in this specification is described in a progressive manner, and the same and similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment. The device embodiment described above is merely illustrative, wherein the modules described as separate components may or may not be physically separated, and the functions of each module can be implemented in the same one or more software and / or hardware when implementing the embodiment of the present disclosure. It is also possible to select some or all of the modules according to actual needs to achieve the purpose of the embodiment. A person of ordinary skill in the art can understand and implement it without paying creative labor.
[0080] The above is only a specific implementation of the embodiment of the present disclosure. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the embodiment of the present disclosure. These improvements and modifications should also be regarded as the protection scope of the embodiment of the present disclosure.
Claims
1. A data processing method, characterized in that: The method comprises: Acquire multiple sets of 3D data collected by the same device source for the target object at different time nodes; For two sets of three-dimensional data at any two time nodes, feature points are extracted from the two sets of three-dimensional data respectively, and the extracted feature points are matched to obtain matching feature point pairs; A target coordinate transformation relationship between the two sets of three-dimensional data is determined based on the feature point pairs, so as to spatially align the two sets of three-dimensional data based on the target coordinate transformation relationship.
2. The method according to claim 1, characterized in that The method further comprises: spatially aligning the multiple sets of three-dimensional data based on the target coordinate transformation relationship between any two sets of three-dimensional data among the multiple sets of three-dimensional data; Based on the alignment results, the changes of the multiple groups of three-dimensional data at different time nodes are displayed.
3. The method according to claim 2, characterized in that Based on the alignment results, the changes of the multiple sets of three-dimensional data at different time nodes are displayed, including: The changes of the multiple groups of three-dimensional data at different time nodes are displayed in the form of animation, wherein the animation frames corresponding to the time points between two adjacent time nodes are generated by interpolation animation.
4. The method according to any one of claims 1 to 3, characterized in that: The determining the target coordinate transformation relationship between the two sets of three-dimensional data based on the feature point pairs includes: Determining an initial coordinate transformation relationship between the two sets of three-dimensional data based on the feature point pairs; Performing semantic segmentation processing on the two groups of three-dimensional data respectively to obtain two semantic entity sets, wherein one semantic entity set corresponds to a group of three-dimensional data; Based on the correspondence between the semantic entities in the two semantic entity sets, the initial coordinate transformation relationship is optimized to obtain a target transformation relationship between the two sets of three-dimensional data.
5. The method according to claim 4, characterized in that The step of optimizing the initial coordinate transformation relationship based on the correspondence relationship between the semantic entities in the two semantic entity sets to obtain a target coordinate transformation relationship includes: Based on the correspondence between the semantic entities in the two semantic entity sets, determining one or more groups of target semantic entity pairs; The initial coordinate transformation relationship is optimized based on the size of the difference between two semantic entities in each group of target semantic entity pairs to obtain the target coordinate transformation relationship.
6. The method according to claim 5, characterized in that In the case where there are invariants in the two semantic entity sets, the target semantic entity pair is a semantic entity pair consisting of the invariants, wherein the invariants are semantic entities whose form does not change with time; In the case where there is no invariant in the two semantic entity sets, the target semantic entity pairs are all semantic entity pairs in the two semantic entity sets, wherein the invariant is a semantic entity whose form does not change with time.
7. The method according to claim 6, characterized in that In the case where the target semantic entity pair is a semantic entity pair formed by the invariant, the difference between two semantic entities in the target semantic entity pair is characterized by a two-norm; In the case that the target semantic entity pairs are all semantic entity pairs in the two semantic entity sets, the difference between the two semantic entities in the target semantic entity pairs is represented by a norm.
8. The method according to claim 4, characterized in that The three-dimensional data is a three-dimensional model of the oral cavity, the semantic entity is a single tooth model, and the corresponding relationship of the semantic entity is determined based on the tooth position number of the tooth model.
9. The method according to any one of claims 1 to 3, characterized in that: Determining the target coordinate transformation relationship of the two sets of three-dimensional data based on the feature point pairs, comprising: determining the target coordinate transformation relationship of the two sets of three-dimensional data based on the difference between the feature point pairs, wherein the difference between the feature point pairs is characterized by a norm; and / or The three-dimensional data is oral CBCT data. Before extracting feature points from two sets of three-dimensional data at any two time nodes, the method further includes: performing isosurface extraction on the CBCT data to obtain surface mesh data.
10. The method according to any one of claims 1 to 3, characterized in that: The feature point pairs include a plurality of groups, and determining the target coordinate transformation relationship between the two groups of three-dimensional data based on the feature point pairs includes: The target transformation relationship is determined based on a weighted average of differences between two feature points in each group of feature point pairs in the multiple groups of feature point pairs, wherein the weight corresponding to the difference between each group of feature point pairs is determined based on the probability and / or magnitude of the change of the object represented by the group of feature point pairs over time.
11. The method according to claim 10, characterized in that The three-dimensional data is three-dimensional facial data, and the feature points include one or more of the following: the center of the eyebrows, the prominent point of the cheekbones, the tip of the nose, and the chin.
12. A data processing device, characterized in that: The device comprises: An acquisition module, used to acquire multiple sets of three-dimensional data collected by the same device source for the target object at different time nodes; A feature point pair determination module is used to extract feature points from two sets of three-dimensional data at any two time nodes, respectively, and match the extracted feature points to obtain matching feature point pairs; An alignment module is used to determine a target coordinate transformation relationship between the two sets of three-dimensional data based on the feature point pairs, so as to spatially align the two sets of three-dimensional data based on the target coordinate transformation relationship.
13. An electronic device, characterized in that: The electronic device includes a processor, a memory, and computer instructions stored in the memory and executable by the processor. When the processor executes the computer instructions, the method according to any one of claims 1 to 11 is implemented.
14. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method according to any one of claims 1 to 11 is implemented.
