Data processing method and apparatus, device, and storage medium
By extracting and matching feature points in the three-dimensional data and determining the coordinate transformation relationship, the accuracy of three-dimensional data alignment at different time nodes is solved, and the accurate alignment and change display of three-dimensional data is achieved.
Patent Information
- Application Number
- PCT/CN2024/090843
- Authority / Receiving Office
- WO · WO
- Patent Type
- Applications
- Current Assignee / Owner
- Priority Date
- 2023-05-09
- Filing Date
- 2024-04-30
- Publication Date
- 2024-11-14
AI Technical Summary
In scenarios such as oral diagnosis and treatment, the three-dimensional data collected at different time nodes need to be spatially aligned to compare the changes in the patient's location. However, it is difficult for the prior art to accurately determine the coordinate transformation relationship, resulting in inaccurate alignment results.
By obtaining the three-dimensional data collected by the same device source at different time nodes, extracting feature points, matching, determining the matching feature point pairs, and determining the target coordinate transformation relationship based on these feature point pairs, thereby aligning the three-dimensional data in space.
It realizes accurate alignment of three-dimensional data at different time nodes, displays data changes, and improves the accuracy and reliability of alignment.
Smart Images

Figure CN2024090843_14112024_PF_FP_ABST
Abstract
Description
Data processing method, device, equipment and storage medium CROSS-REFERENCE TO RELATED APPLICATIONS
[0000] This application claims priority to the Chinese patent application filed with the China Patent Office on May 9, 2023, with application number 202310521594.X, and invention name “Data processing method, device, equipment and storage medium”, all contents of which are incorporated by reference in this application. Technical Field The present disclosure relates to the field of data processing technology, and in particular to a data processing method, device, equipment and storage medium. Background Art In some scenarios, it is necessary to determine the coordinate transformation relationship of the three-dimensional data collected by the same device source for the same target object at different time nodes, so as to spatially align the three-dimensional data collected at different time nodes. Taking the oral diagnosis and treatment scenario as an example, during the oral diagnosis and treatment process, three-dimensional data such as three-dimensional scanning data and CBCT (Cone-Beam Computed Tomography) data of the patient's oral cavity, face and other parts are usually collected. In order to facilitate the comparison of changes in various parts of the patient (for example, teeth) at different time nodes, the three-dimensional data collected at different time nodes need to be spatially aligned. Therefore, it is necessary to provide a solution that can accurately determine the coordinate transformation relationship of the three-dimensional data collected at different time nodes, so that the three-dimensional data collected at different time nodes can be displayed after being aligned in space. Summary of the invention The present disclosure provides a data processing method, apparatus, device and storage medium. According to a first aspect of an embodiment of the present disclosure, a data processing method is provided, the method comprising: acquiring multiple sets of three-dimensional data collected by the same device source for a target object at different time nodes; for any two sets of three-dimensional data, respectively extracting feature points from the two sets of three-dimensional data, matching the extracted feature points to obtain matching feature point pairs; and determining a target coordinate transformation relationship between the two sets of three-dimensional data based on the feature point pairs. According to a second aspect of an embodiment of the present disclosure, a data processing device is provided, comprising: an acquisition module, configured to acquire multiple sets of three-dimensional data collected by the same device source for a target object at different time nodes; a feature point pair determination module, configured to extract feature points from any two sets of three-dimensional data, respectively, match the extracted feature points, and obtain matched feature point pairs; and an alignment module, configured to determine a target coordinate transformation relationship between the two sets of three-dimensional data based on the feature point pairs. According to a third aspect of the embodiments of the present disclosure, there is provided an electronic device, the electronic device comprising a processor, a memory A memory, computer instructions stored in the memory and executable by the processor, when the processor executes the computer instructions, the method mentioned in the first aspect can be implemented. According to a fourth aspect of an embodiment of the present disclosure, a computer-readable storage medium is provided, on which computer instructions are stored. When the computer instructions are executed, the method mentioned in the first aspect is implemented. In the disclosed embodiment, after obtaining any two sets of three-dimensional data collected by the same device source at different time nodes for the same target object, feature points can be extracted from the two sets of three-dimensional data respectively, and the extracted feature points can be matched to obtain matching feature point pairs, and then the target coordinate transformation relationship of the two sets of three-dimensional data can be determined based on these matching feature point pairs, so that the two sets of three-dimensional data can be spatially aligned and displayed based on the target coordinate transformation relationship. In this way, the three-dimensional data collected for the same object at different time nodes can be aligned to display the changes in the three-dimensional data collected at different time nodes. It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the disclosure. BRIEF DESCRIPTION OF THE DRAWINGS The drawings herein are incorporated into the specification and constitute a part of the specification. These drawings illustrate embodiments consistent with the present disclosure and are used to illustrate the technical solutions of the present disclosure together with the specification. FIG. 1 is a flow chart of data processing according to an embodiment of the present disclosure. FIG. 2 is a schematic diagram of three-dimensional data alignment according to an embodiment of the present disclosure. FIG. 3 is a schematic diagram of three-dimensional data alignment according to an embodiment of the present disclosure. FIG. 4 is a schematic diagram of the logical structure of a data processing device according to an embodiment of the present disclosure. FIG. 5 is a schematic diagram of a logical structure of a device according to an embodiment of the present disclosure. DETAILED DESCRIPTION Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present disclosure. Instead, they are merely examples of devices and methods consistent with some aspects of the present disclosure as detailed in the appended claims. The terms used in this disclosure are for the purpose of describing specific embodiments only and are not intended to limit the disclosure. The singular forms "a," "an," and "the" used in this disclosure and the appended claims are intended to include the plural forms as well, unless the context clearly indicates otherwise. It should also be understood that the term "and / or" as used herein refers to and Contains any or all possible combinations of one or more associated listed items. In addition, the term "at least one" herein means any combination of at least two of any one or more of a plurality of items. It should be understood that although the terms first, second, third, etc. may be used in the present disclosure to describe various information, such information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of the present disclosure, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information. Depending on the context, the word "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determining". In order to enable those skilled in the art to better understand the technical solutions in the embodiments of the present disclosure and to make the above-mentioned purposes, features and advantages of the embodiments of the present disclosure more obvious and understandable, the technical solutions in the embodiments of the present disclosure are further described in detail below in conjunction with the accompanying drawings. In some scenarios, it is necessary to determine the coordinate transformation relationship of the three-dimensional data collected by the same device source for the same target object at different time nodes, so as to spatially align the three-dimensional data collected at different time nodes. Taking the oral diagnosis and treatment scenario as an example, during the oral diagnosis and treatment process, three-dimensional data such as three-dimensional scanning data and CBCT (Cone-Beam Computed Tomography) data of the patient's oral cavity, face and other parts are usually collected. In order to facilitate the comparison of changes in various parts of the patient at different time nodes, the three-dimensional data collected at different time nodes need to be spatially aligned. For example, doctors usually need to compare the changes in the patient's teeth at different time nodes, so they need to spatially align the three-dimensional data of the teeth that are not collected through the time nodes. Therefore, it is necessary to provide a solution that can accurately determine the coordinate transformation relationship of the three-dimensional data collected at different time nodes, so that the three-dimensional data collected at different time nodes can be displayed after being aligned in space. Based on this, the embodiment of the present disclosure provides a data processing method. After obtaining any two sets of three-dimensional data collected by the same device source for the same target object at different time nodes, feature points can be extracted from the two sets of three-dimensional data respectively, and the extracted feature points can be matched to obtain matched feature point pairs. Then, the target coordinate transformation relationship between the two sets of three-dimensional data can be determined based on these matched feature point pairs, so that any two sets of three-dimensional data can be spatially aligned and displayed based on the target coordinate transformation relationship. The data processing method provided by the embodiments of the present disclosure can be implemented by a self-developed software code with data processing or viewing functions. For example, in some scenarios, the software code can be presented in the form of data viewing software, or can also be presented in the form of a web client with data viewing functions. Users can import the three-dimensional data collected by the same device source for the same target object at different time nodes into the data viewing software or web client, and the data viewing software or web client can align the imported three-dimensional data and display it. The target object mentioned in the embodiments of the present disclosure may be various objects that require three-dimensional data collection and display. For example, the target object may be the patient's oral cavity, face, etc. The three-dimensional data may be the patient's oral cavity three-dimensional scanning data, facial three-dimensional scanning data, oral CBCT data, skull CBCT data, etc., and the embodiments of the present disclosure are not limited thereto. Specifically, as shown in FIG1 , the data processing method provided by the embodiment of the present disclosure includes the following steps: S102, obtaining multiple sets of three-dimensional data collected by the same device source for the target object at different time nodes; In step S102, multiple sets of three-dimensional data collected by the same device source for the target object at different time nodes can be obtained. For example, taking the execution subject of the method as data viewing software, the user can import two or more sets of three-dimensional data collected by the same device source for the same target object at different time nodes in the data viewing software. Among them, the three-dimensional data can be three-dimensional point cloud data, three-dimensional mesh data, or volume data, etc., which is not limited by the embodiments of the present disclosure. S104, for any two sets of three-dimensional data, extracting feature points from the two sets of three-dimensional data respectively, matching the extracted feature points, and obtaining matching feature point pairs; In step S104, for any two groups of three-dimensional data in the above-mentioned multiple groups of three-dimensional data, feature points can be extracted from the two groups of three-dimensional data respectively, wherein the feature points can be geometric feature points or texture feature points in the three-dimensional model represented by the three-dimensional data (for example, the three-dimensional model is a three-dimensional model carrying texture information), for example, the feature points can be some three-dimensional points with large geometric shape changes in the three-dimensional model, for example, they can be contour points; or some three-dimensional points with large texture changes. After extracting the respective feature points from the two groups of three-dimensional data, the feature points extracted from the two groups of three-dimensional data can be matched to obtain one or more groups of matched feature point pairs. Among them, the two feature points in the matched feature point pair correspond to the same three-dimensional point in the three-dimensional space, for example, the two feature points in the feature point pair both represent the center of the eyebrows, or the two feature points in the feature point pair both represent the protruding points of the zygomatic bones. After extracting the feature points of the two sets of three-dimensional data, the matching feature point pairs may be determined based on the Ransca algorithm, or a similar algorithm may be used, which is not limited in the embodiments of the present disclosure. S106: Determine a target coordinate transformation relationship between the two sets of three-dimensional data based on the feature point pairs. In step S106, after determining the matched feature point pairs, a target coordinate transformation relationship representing the position transformation relationship of the two sets of three-dimensional data can be determined based on the matched feature point pairs, so as to align the two sets of three-dimensional data in space using the target coordinate transformation relationship. For example, if the two sets of three-dimensional data are two three-dimensional models, the rigid body transformation matrix of the two three-dimensional models can be calculated, which is the target coordinate transformation relationship. For any two sets of three-dimensional data among the multiple sets of three-dimensional data, the above method can be used to determine the two sets of three-dimensional data. The target coordinate transformation relationship of the three-dimensional data can ultimately be used to align the multiple sets of three-dimensional data in space based on the target coordinate transformation relationship. In some embodiments, in order to display the changes of three-dimensional data collected at different time nodes, after the multiple sets of three-dimensional data are spatially aligned based on the target coordinate transformation relationship between any two sets of three-dimensional data in the multiple sets of three-dimensional data, the changes of the multiple sets of three-dimensional data at different time nodes can be further displayed based on the alignment results. In some embodiments, in order to better display the changes of the three-dimensional data collected by the same device source at different time nodes, the changes of the three-dimensional data collected by the same device source at different time nodes can be displayed in the form of animation. Since the acquisition process is discrete, the animation frames corresponding to the time points between two adjacent time nodes can be generated by interpolation animation. For example, continuous animation frames can be generated by Morphing technology to continuously present the changes of the three-dimensional data between adjacent time nodes. Of course, in some scenarios, multiple sets of three-dimensional data collected at different time nodes can be superimposed together. Since the overlapping parts will overlap after superposition, the changes in non-overlapping areas (i.e., changes) can be highlighted. As the target object changes over time, the shape of the target object will change to some extent. For example, taking the oral cavity as an example, the shape and arrangement of the teeth will change to some extent over time. Therefore, it is difficult to establish an accurate correspondence between the feature points, so that the coordinate transformation relationship of the two sets of three-dimensional data determined based only on the matched feature point pairs may not be accurate, resulting in inaccurate final alignment results. In order to obtain a more accurate coordinate transformation relationship, in some embodiments, as shown in FIG2, feature points (feature point 1, feature point 2) can be extracted from two sets of three-dimensional data (three-dimensional data 1, three-dimensional data 2) collected at different time nodes (for example, time node 1, time node 2), and then the feature points are matched to obtain feature point pairs, and the initial coordinate transformation relationship of the two sets of three-dimensional data is determined based on the feature point pairs. Through the initial coordinate transformation relationship, the two sets of three-dimensional data can be roughly aligned. After completing the rough alignment, the two sets of three-dimensional data can be semantically segmented to obtain two semantic entity sets (for example, semantic entity set 1, semantic entity set 2), wherein one semantic entity set corresponds to a set of three-dimensional data, that is, the semantic entities in the set are obtained by semantic segmentation of the set of three-dimensional data. Since both sets of three-dimensional data represent the same target object, there is a one-to-one correspondence between the semantic entities in the two semantic entity sets, that is, the two semantic entities in the two semantic entity sets that represent the same object in three-dimensional space can be associated. Furthermore, based on the correspondence between the semantic entities in the two semantic entity sets, the semantic entity pairs representing the same object can be determined, and the initial coordinate transformation relationship can be optimized based on these semantic entity pairs to obtain a more accurate target coordinate transformation relationship, so as to use the target coordinate transformation relationship to precisely align the two sets of three-dimensional data. Compared with the correspondence between the feature points in the two sets of three-dimensional data, the correspondence between the semantic entities in the two sets of three-dimensional data is more accurate, and the correspondence between the semantic entities can be used. By optimizing the initial coordinate transformation relationship, a more accurate coordinate transformation relationship can be obtained. In some embodiments, considering that the difference between two semantic entities representing the same object in three-dimensional space is usually small, therefore, when optimizing the initial coordinate transformation relationship based on the correspondence relationship between the semantic entities in the two semantic entity sets to obtain the target coordinate transformation relationship, one or more groups of target semantic entity pairs can be determined based on the correspondence relationship between the semantic entities in the two semantic entity sets, wherein the two semantic entities in the semantic entity pairs represent the same object in three-dimensional space. Then, the initial coordinate transformation relationship can be optimized based on the size of the difference between the two semantic entities in each group of target semantic entity pairs to obtain the target coordinate transformation relationship. For example, considering that the difference between two semantic entities representing the same object in three-dimensional space is generally small, therefore, when determining the target coordinate transformation relationship, the following optimization conditions can be set: solve the best target coordinate transformation relationship so that after the two semantic entities in each group of target semantic entity pairs are aligned in space using the target coordinate transformation relationship, the cumulative value of the difference between the two semantic entities in the one or more groups of target semantic entity pairs is as small as possible. Of course, considering that different semantic entities may change over time, resulting in a large difference between the two semantic entities in the same semantic entity pair. When optimizing the initial coordinate transformation relationship using semantic entity pairs, the difference between the two semantic entities in the same semantic entity pair is usually minimized as the optimization target to obtain the best target coordinate transformation relationship. Therefore, if the initial coordinate transformation relationship is optimized using semantic entity pairs that change over time, since the two semantic entities of the semantic entity pair may have large differences, the target coordinate transformation relationship finally determined will be inaccurate. Therefore, in some embodiments, as shown in FIG3, in order to obtain a more accurate target coordinate transformation relationship, the target coordinate transformation relationship can be determined by using semantic entity pairs composed of semantic entities that do not change over time. For example, it can be determined whether there are invariants in the two semantic entity sets, where the invariants are semantic entities whose morphology does not change over time. For example, taking the oral cavity as an example, some teeth in the oral cavity will not change over time, so these teeth can be used as invariants. Among them, the invariants can be pre-set by the user based on prior knowledge (for example, oral diagnosis and treatment results, experience, etc.). If there are invariants in two semantic entity sets, the invariants with corresponding relationships in the two semantic entity sets can be used to form a target semantic entity pair, and then the target semantic entity pair can be used to optimize the initial coordinate transformation relationship. By first determining the invariants from the semantic entities obtained by semantic segmentation of two sets of three-dimensional data, and then using the invariants with corresponding relationships to form a target semantic entity pair, since the two semantic entities of the target semantic entity pair have not changed, that is, the difference is basically 0, and then the difference between the two semantic entities in the target semantic entity pair is as small as possible as a constraint condition, the target coordinate transformation relationship is determined, thereby obtaining a more accurate target coordinate transformation relationship. In some embodiments, the three-dimensional data collected at different time points may not have invariants. For example, taking the oral cavity as an example, as time changes, all teeth in the oral cavity have changes in geometry or arrangement posture. Under this circumstance, it is impossible to find invariants from the semantic entity sets of the two sets of three-dimensional data. Therefore, all the semantic entity pairs with corresponding relationships in the two semantic entity sets can be used as target semantic entity pairs. In some embodiments, if there is an invariant in the semantic entity set, that is, the target semantic entity pair is a semantic entity pair composed of invariants, in this case, the error is small, and the difference between the two semantic entities in the target semantic entity pair can be represented by the two-norm. In some embodiments, if there is no invariant in the semantic entity set, that is, the target semantic entity pairs are all semantic entity pairs in the two semantic entity sets, since the two semantic entities in the target semantic entity pair have changed over time, there are differences between the two, which will cause errors. In order to concentrate the errors in a limited area, when characterizing the differences between the two semantic entities in the target semantic entity pair, a norm can be used to characterize them. For scenarios where there is no invariant in the semantic entity set, considering that there are many mismatched point pairs, a norm is used to characterize the differences to optimize the matching results, so that a more accurate coordinate transformation relationship can be obtained. In some embodiments, the three-dimensional data may be a three-dimensional model of the oral cavity, and the semantic entity may be a single tooth model. The correspondence between the tooth models in the two three-dimensional models collected at different time points may be determined based on the tooth position identification (e.g., tooth position number) of the teeth. For example, for two sets of oral three-dimensional model data collected at different time points, semantic segmentation processing may be performed on them to obtain each tooth model and the tooth position number corresponding to the tooth model, and then based on the tooth position number of each tooth model in the two three-dimensional models, the correspondence between the teeth in the two three-dimensional models is established. For example, take the alignment of three-dimensional data collected from the oral cavity of the same patient at different time points as an example. Assume that the three-dimensional data records collected by the oral scanning device at different time points are as follows: in, is the maxillary data collected at time t (the superscript down represents the mandibular data), Represents the three-dimensional tooth model of the i-th tooth at time t. For each oral three-dimensional model at a certain time, semantic segmentation can be performed to obtain multiple tooth models, and then the tooth position number of each tooth model can be determined by tooth position recognition. The tooth models in the oral three-dimensional model at two time nodes can be associated through the tooth position number to establish a corresponding relationship between the tooth models. The steps of aligning two sets of 3D data are divided into coarse alignment and fine alignment: (1) Rough alignment: for t i ,t j The three-dimensional data of the oral model collected at all times and Feature points are extracted separately to obtain geometric feature points, and the extracted set feature points are matched using the Ransca algorithm to obtain matching feature point pairs, and then the rigid body transformation matrix T of the two sets of three-dimensional data is calculated based on the feature point pairs. i,j . (2) Fine alignment: Through the above rough alignment steps, and A preliminary alignment has been done within the same coordinate system. (a) Since the teeth collected at different times may change (such as tooth morphology, tooth arrangement posture, tooth loss, etc.), it is impossible to establish a correspondence relationship in the entire domain (establish a correspondence relationship between three-dimensional points). Therefore, the treatment plan can be used to screen out t i ,t j A collection of teeth that remain unchanged at all times Among them, k represents the identifier of the tooth with unchanged shape. The tooth models in this set do not have changes in geometric shape and arrangement posture, so they can be considered as t i ,t j The key to aligning three-dimensional models is to find the accurate corresponding points between three-dimensional models. Therefore, the corresponding point pairs on the same tooth number in the tooth set are used to complete the precise alignment, that is, the best T can be determined by formula (1), and the final rigid body transformation matrix of the two sets of three-dimensional data is TTi,j: Where K represents the tooth model with unchanged morphology. (b) Of course, in some scenarios, t i →t jAll teeth have changes in geometry or arrangement at all times, so there is no actual invariant. The 1-norm energy can be used to measure the difference between tooth models, so that the error is concentrated in a limited area. For example, the optimal T can be determined by formula (2), and the final rigid body transformation matrix of the two sets of three-dimensional data is TTi,j: Where K represents all tooth models. In some embodiments, the three-dimensional data may be oral CBCT data. Since CBCT is volume data, before aligning two sets of CBCT data collected at different time points, isosurface extraction may be performed on the CBCT data to obtain surface mesh data. Feature points are then extracted from the surface mesh data, and the target coordinate transformation relationship is determined based on the feature point pairs. In some embodiments, when determining the target coordinate transformation relationship between two sets of three-dimensional data based on feature point pairs, it is considered that target objects such as the mouth and face tend to deform over time, and thus the obtained matching feature point pairs often have many mismatches. In this case, using the classical least squares method to obtain the target coordinate transformation relationship will not be very accurate. In order to obtain a more accurate target coordinate transformation relationship, when determining the target coordinate transformation relationship using feature point pairs, a norm can be used to characterize the difference between the two feature points in the feature point pair, and then the rigid body transformation matrix of the two sets of three-dimensional data is optimized based on the difference between the two feature points of the feature point pair to obtain the target coordinate transformation relationship. By using a norm to characterize the difference between feature point pairs, the final optimization result can be made more accurate and reliable. In some embodiments, the feature point pair may include multiple groups (multiple groups). When determining the target coordinate transformation relationship of two groups of three-dimensional data based on the feature point pair, the target coordinate change relationship can be solved based on the difference between the two feature points in the feature point pair being as small as possible as a constraint condition. Considering that different types of feature points have different probabilities of change or different magnitudes of change over time. Therefore, when determining the target coordinate change relationship based on the difference between the two feature points in the feature point pair, the target coordinate transformation relationship can be determined based on the weighted average of the difference between the two feature points in each group (each) of feature point pairs. Among them, considering that the probability of different feature points changing over time is different, different weights can be set for the differences of different feature point pairs. For example, for feature points that are likely to change over time, or feature points that will change greatly over time, the weight of their differences can be set smaller, and vice versa, they can be set larger. In some embodiments, the three-dimensional data is three-dimensional facial data, and the feature points may include one or more of the following: the center of the eyebrows, the prominent point of the cheekbones, the tip of the nose, the chin, etc. For example, take the alignment of facial 3D data collected at different time points as an example. The facial 3D data collected by the facial scanning device at different time points is recorded as M t , feature points can be extracted from each face 3D data at different time nodes to obtain a set of feature points, such as eyes, nose tip, chin and other facial feature points The corresponding relationship between the feature points in the three-dimensional face data at different time nodes is established to obtain the matching feature point pairs. Then, the rigid body transformation matrix Ti,j of the two sets of three-dimensional data can be optimized based on the difference between the matching feature point pairs. Specifically, as shown in formula (3): Among them, ω k is the weight corresponding to each feature point, is the Kth feature point at time ti, is the Kth feature point at time tj. Among them, considering that different types of feature points, such as eyes, nose tip, and chin, have different probabilities of changing over time, different weights can be set for the differences corresponding to different types of feature points. The weights of feature points with a higher probability of change can be smaller, and vice versa. In this way, the determined rigid body transformation matrix Ti,j is more accurate. It is not difficult to understand that the solutions described in the above embodiments can be combined when there is no conflict, and they are not listed one by one in the embodiments of the present disclosure. Accordingly, the embodiment of the present disclosure further provides a data processing device, as shown in FIG4 , wherein the device 40 includes: An acquisition module 41 is used to acquire multiple sets of three-dimensional data collected by the same device source for the target object at different time nodes; A feature point pair determination module 42 is used to extract feature points from any two sets of three-dimensional data, respectively, and match the extracted feature points to obtain matching feature point pairs; The alignment module 43 is used to determine the target coordinate transformation relationship between the two sets of three-dimensional data based on the feature point pairs. Among them, the specific steps of the above-mentioned device executing the task processing method can refer to the description in the above-mentioned method embodiment, and will not be repeated here. Furthermore, an embodiment of the present disclosure also provides a device, as shown in Figure 5, which includes a processor 51, a memory 52, and computer instructions stored in the memory 52 for execution by the processor 51, and the processor 51 implements any of the methods described in the above embodiments when executing the computer instructions. The present disclosure also provides a computer-readable storage medium on which a computer program is stored. When the program is executed by a processor, the method described in any of the above embodiments is implemented. Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, modules of programs or other data. Examples of computer storage media include, but are not limited to, Phase Change Memory (Phase Random Access Memory, PRAM), Static Random Access Memory (Static Random Access Memory, SRAM), Dynamic Random Access Memory (Dynamic Random Access Memory, DRAM), other types of Random Access Memory (Random Access Memory, RAM), Read-Only Memory (Read-Only Memory, ROM), Electrically Erasable Programmable Read Only Memory (EEPROM), Flash memory or other memory technology, Compact Disc Read-Only Memory (CD-ROM), Digital Versatile Disc (Digital Versatile Disc, DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission medium that can be used to store information that can be accessed by a computing device. According to the definition in this article, computer-readable media does not include temporary computer-readable media (transitory media), such as modulated data signals and carrier waves. It can be seen from the above description of the implementation method that those skilled in the art can clearly understand that the embodiments of the present disclosure can be implemented by means of software plus a necessary general hardware platform. Based on this understanding, the technical solution of the embodiments of the present disclosure, or the part that contributes to the prior art, can be embodied in the form of a software product. The computer software product can be stored in a storage medium, such as ROM / RAM, a disk, an optical disk, etc., including several instructions. It is used to enable a computer device (which may be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments of the present disclosure. The systems, devices, modules or units described in the above embodiments may be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer, which may be in the form of a personal computer, a laptop computer, a cellular phone, a camera phone, a smart phone, a personal digital assistant, a media player, a navigation device, an email transceiver, a game console, a tablet computer, a wearable device or a combination of any of these devices. Each embodiment in this specification is described in a progressive manner, and the same and similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment. The device embodiment described above is merely illustrative, wherein the modules described as separate components may or may not be physically separated, and the functions of each module can be implemented in the same one or more software and / or hardware when implementing the embodiment of the present disclosure. It is also possible to select some or all of the modules according to actual needs to achieve the purpose of the embodiment. A person of ordinary skill in the art can understand and implement it without paying creative labor. Industrial Applicability The data processing method provided by the embodiment of the present disclosure can align three-dimensional data collected for the same object at different time nodes so as to display the changes in the three-dimensional data collected at different time nodes, and has strong industrial applicability.
Claims
1. A data processing method, the method include: Acquire multiple sets of 3D data collected by the same device source for the target object at different time nodes; For any two sets of three-dimensional data, Extracting feature points from the two sets of three-dimensional data respectively; Match the extracted feature points to obtain matching feature point pairs; The target coordinate transformation relationship between the two sets of three-dimensional data is determined based on the feature point pairs.
2. The method according to claim 1, further comprising: include: Based on the target coordinate transformation relationship between any two sets of three-dimensional data among the multiple sets of three-dimensional data, aligning the two sets of three-dimensional data in space; Based on the alignment results of the multiple sets of three-dimensional data, the changes of the multiple sets of three-dimensional data at different time nodes are displayed.
3. The method according to claim 2, based on the alignment results of the multiple sets of three-dimensional data, displays the changes of the multiple sets of three-dimensional data at different time nodes, include: The changes of the multiple groups of three-dimensional data at different time nodes are displayed in the form of animation, wherein the animation frames corresponding to the time points between two adjacent time nodes are generated by interpolation animation.
4. The method according to any one of claims 1 to 3, wherein the target coordinate transformation relationship of the two sets of three-dimensional data is determined based on the feature point pairs. include: Determining an initial coordinate transformation relationship between the two sets of three-dimensional data based on the feature point pairs; Performing semantic segmentation processing on the two groups of three-dimensional data respectively to obtain two semantic entity sets, wherein one semantic entity set corresponds to a group of three-dimensional data; Based on the correspondence between the semantic entities in the two semantic entity sets, the initial coordinate transformation relationship is optimized to obtain a target transformation relationship between the two sets of three-dimensional data.
5. The method according to claim 4, wherein the initial coordinate transformation relationship is optimized based on the correspondence relationship between the semantic entities in the two semantic entity sets to obtain a target coordinate transformation relationship, include: Based on the correspondence between the semantic entities in the two semantic entity sets, determining one or more groups of target semantic entity pairs; The initial coordinate transformation relationship is optimized based on the size of the difference between two semantic entities in each group of target semantic entity pairs to obtain the target coordinate transformation relationship.
6. The method according to claim 5, When there are invariants in the two semantic entity sets, the target semantic entity pair is the invariant The semantic entity pairs composed of in, The invariant is a semantic entity whose form does not change with time; In the case that the invariant does not exist in the two semantic entity sets, the target semantic entity pairs are all semantic entity pairs in the two semantic entity sets.
7. The method according to claim 6, In the case where the target semantic entity pair is a semantic entity pair formed by the invariant, the difference between two semantic entities in the target semantic entity pair is characterized by a two-norm; In the case that the target semantic entity pairs are all semantic entity pairs in the two semantic entity sets, the difference between the two semantic entities in the target semantic entity pairs is represented by a norm.
8. According to the method of claim 4, the three-dimensional data is a three-dimensional model of the oral cavity, the semantic entity is a single tooth model, and the corresponding relationship of the semantic entity is determined based on the tooth position number of the tooth model.
9. The method according to any one of claims 1 to 3, wherein the target coordinate transformation relationship between the two sets of three-dimensional data is determined based on the feature point pairs. include: The target coordinate transformation relationship between the two sets of three-dimensional data is determined based on the difference between the feature point pairs, wherein the difference between the feature point pairs is represented by a norm.
10. The method according to any one of claims 1 to 9, wherein the three-dimensional data is oral CBCT data, and before extracting feature points from any two sets of three-dimensional data, respectively, the method further comprises: include: Isosurface extraction is performed on the CBCT data to obtain surface mesh data.
11. The method according to any one of claims 1 to 3, wherein the feature point pairs include a plurality of feature point pairs, and the target coordinate transformation relationship between the two sets of three-dimensional data is determined based on the feature point pairs. include: The target transformation relationship is determined based on a weighted average of differences between two feature points in each feature point pair in the multiple feature point pairs, wherein the weight corresponding to the difference between the two feature points in each feature point pair is determined based on a probability and / or magnitude of a change in an object represented by the feature point pair over time.
12. The method according to claim 11, wherein the three-dimensional data is three-dimensional facial data, and the feature points include one or more of the following: the center of the eyebrows, the prominent point of the cheekbones, the tip of the nose, and the chin.
13. A data processing device, the device include: An acquisition module is configured to acquire multiple sets of three-dimensional data collected by the same device source for the target object at different time nodes; A feature point pair determination module is configured to extract feature points from any two sets of three-dimensional data, respectively, and match the extracted feature points to obtain matching feature point pairs; The alignment module is configured to determine the target coordinate transformation relationship between the two sets of three-dimensional data based on the feature point pairs.
14. An electronic device, comprising a processor, a memory, and computer instructions stored in the memory and executable by the processor, wherein the processor implements the method according to any one of claims 1 to 12 when executing the computer instructions.
15. A computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the method according to any one of claims 1 to 12.
Citation Information
Patent Citations
Infrared photography-based bio-feature 3D four-dimensional data identification method and system
CN108564018A
electric power line patrol LiDAR data automatic registration method based on tower characteristic points
CN109872352A
Image detection method and device and electronic equipment
CN112396117A
Three-dimensional dynamic tracking method and device, electronic equipment and storage medium
CN113516781A
Three-dimensional point cloud processing method and device, equipment and storage medium
CN114926549A
