Morphology and posture estimation method, device, equipment and storage medium
Through the combination of three-dimensional cameras and neural network models, the problem of low accuracy of morphology and posture prediction of organ or biological tissues is solved, high-precision morphology and posture prediction is achieved, surgical risks are reduced, and real-time internal structure information is provided.
Patent Information
- Application Number
- CN202010342911.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2020-04-27
- Publication Date
- 2024-11-26
- Estimated Expiration
- 2040-05-11
AI Technical Summary
In the morphology and posture prediction of organ or biological tissues, the prediction accuracy is low due to deformation and occlusion of organ or biological tissues, and it is difficult to update internal structural information in real time, increasing the stress and surgical risks of doctors.
By using a three-dimensional camera to collect surface three-dimensional morphological information of organs or biological tissues, combine preset matching functions and finite element-based neural network model to perform morphology and posture prediction, and estimate internal structure deformation based on surface morphology information to improve the estimate Accuracy.
Achieve high-precision prediction of organ or biological tissue morphology and posture in open and endoscopic surgery, reduces doctors' stress and surgical risks, provides real-time internal structural information, and improves the safety and accuracy of the surgery. .
Smart Images

Figure CN113643433B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of data processing technology, and in particular to a form and posture estimation method, device, equipment and storage medium. Background Art
[0002] Whether in open surgery or endoscopic surgery, if the doctor can more accurately obtain the morphology and posture information of the current surgical object's organs or biological tissues, it will greatly reduce the doctor's stress and surgical risks.
[0003] At present, the morphology and posture estimation of organs or biological tissues is mainly based on limited feature points or partial outlines. Affected by the deformation and occlusion of organs or biological tissues, the morphology and posture estimation of organs or biological tissues based on limited feature points or partial outlines has low estimation accuracy. Summary of the invention
[0004] Embodiments of the present invention provide a morphology and posture estimation method, apparatus, device and storage medium, which can use the surface three-dimensional morphology information of an organ or biological tissue to estimate the morphology and posture of the organ or biological tissue, thereby improving the estimation accuracy.
[0005] On the one hand, an embodiment of the present invention provides a form and posture estimation method, the method comprising:
[0006] Using a three-dimensional camera to collect three-dimensional surface morphological information of a target object, wherein the target object includes a target organ or a target biological tissue;
[0007] Obtaining an initial three-dimensional model corresponding to the target object;
[0008] The preset matching function is used to match the surface 3D morphological information with the initial 3D model to estimate the shape and posture of the target object.
[0009] In one embodiment of the present invention, the expression corresponding to the preset matching function is:
[0010] (V',x,y,z,α,β,θ)=H(V,So)
[0011] Among them, V' is the estimated shape of the target object, (x, y, z) is the coordinate information included in the estimated posture of the target object, (α, β, θ) is the rotation pointing angle included in the estimated posture of the target object, V is the initial three-dimensional model corresponding to the target object, So is the surface three-dimensional morphology information of the target object, H is the corresponding rule of the preset matching function, and the preset matching function aims to minimize the difference between So and V'.
[0012] In one embodiment of the present invention, the form and posture estimation method provided by the embodiment of the present invention may also include:
[0013] Estimate the internal structure deformation of the target object based on the surface three-dimensional morphology information.
[0014] In one embodiment of the present invention, estimating the internal structural deformation of the target object based on the surface three-dimensional morphological information includes:
[0015] Based on the three-dimensional surface morphology information, the neural network model is used to estimate the internal structure deformation of the target object.
[0016] In one embodiment of the present invention, the neural network model is a finite element based neural network model.
[0017] In one embodiment of the present invention, estimating the internal structural deformation of the target object based on the surface three-dimensional morphological information includes:
[0018] Determine the change range of the surface of the target object according to the three-dimensional surface morphology information obtained twice adjacently;
[0019] According to the magnitude of the change, the internal structural deformation of the target object is estimated.
[0020] In one embodiment of the present invention, estimating the internal structural deformation of the target object according to the variation amplitude includes:
[0021] If the change amplitude is smaller than the preset first threshold, it is estimated that the internal structure of the target object has not been deformed.
[0022] In one embodiment of the present invention, estimating the internal structural deformation of the target object according to the variation amplitude includes:
[0023] If the change amplitude is greater than a preset second threshold, the change volume corresponding to the change of the surface of the target object is calculated according to the surface three-dimensional morphology information acquired twice adjacently;
[0024] The changed volume is determined as the internal structure deformation of the target object.
[0025] On the other hand, an embodiment of the present invention provides a form and posture estimation device, the device comprising:
[0026] A collection module, used for collecting the surface three-dimensional morphological information of the target object using a three-dimensional camera, wherein the target object includes a target organ or a target biological tissue;
[0027] An acquisition module, used to acquire an initial three-dimensional model corresponding to the target object;
[0028] The estimation module is used to match the surface three-dimensional morphological information with the initial three-dimensional model using a preset matching function to estimate the shape and posture of the target object.
[0029] In one embodiment of the present invention, the expression corresponding to the preset matching function is:
[0030] (V',x,y,z,α,β,θ)=H(V,So)
[0031] Among them, V' is the estimated shape of the target object, (x, y, z) is the coordinate information included in the estimated posture of the target object, (α, β, θ) is the rotation pointing angle included in the estimated posture of the target object, V is the initial three-dimensional model corresponding to the target object, So is the surface three-dimensional morphology information of the target object, H is the corresponding rule of the preset matching function, and the preset matching function aims to minimize the difference between So and V'.
[0032] In one embodiment of the present invention, the form and posture estimation device provided by the embodiment of the present invention may also include:
[0033] The internal structure deformation estimation module is used to estimate the internal structure deformation of the target object based on the surface three-dimensional morphology information.
[0034] In one embodiment of the present invention, the internal structure deformation estimation module can be specifically used for:
[0035] Based on the three-dimensional surface morphology information, the neural network model is used to estimate the internal structure deformation of the target object.
[0036] In one embodiment of the present invention, the neural network model is a finite element based neural network model.
[0037] In one embodiment of the present invention, the internal structure deformation estimation module may include:
[0038] A change range determination unit, used to determine the change range of the surface of the target object according to the three-dimensional surface morphology information acquired twice adjacently;
[0039] The internal structure deformation estimation unit is used to estimate the internal structure deformation of the target object according to the change amplitude.
[0040] In one embodiment of the present invention, the internal structure deformation estimation unit can be specifically used for:
[0041] If the change amplitude is smaller than the preset first threshold, it is estimated that the internal structure of the target object has not been deformed.
[0042] In one embodiment of the present invention, the internal structure deformation estimation unit can be specifically used for:
[0043] If the change amplitude is greater than a preset second threshold, the change volume corresponding to the change of the surface of the target object is calculated according to the surface three-dimensional morphology information acquired twice adjacently;
[0044] The changed volume is determined as the internal structure deformation of the target object.
[0045] In another aspect, an embodiment of the present invention provides a morphology and posture estimation device, the device comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor;
[0046] When the processor executes the computer program, the form and posture estimation method provided by the embodiment of the present invention is implemented.
[0047] On the other hand, an embodiment of the present invention provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the form and posture estimation method provided by the embodiment of the present invention is implemented.
[0048] The morphology and posture estimation method, device, equipment and storage medium of the embodiments of the present invention can use the surface three-dimensional morphology information of the organ or biological tissue to estimate the morphology and posture of the organ or biological tissue, and is not affected by the deformation, occlusion, etc. of the organ or biological tissue, and can improve the estimation accuracy. BRIEF DESCRIPTION OF THE DRAWINGS
[0049] In order to more clearly illustrate the technical solution of the embodiment of the present invention, the following briefly introduces the drawings required for use in the embodiment of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.
[0050] Figure 1 A first schematic flow chart of a method for estimating a form and posture provided by an embodiment of the present invention is shown;
[0051] Figure 2 A second schematic flow chart of the form and posture estimation method provided by an embodiment of the present invention is shown;
[0052] Figure 3 A first structural schematic diagram of a morphology and posture estimation device provided by an embodiment of the present invention is shown;
[0053] Figure 4 A second structural schematic diagram of the form and posture estimation device provided by an embodiment of the present invention is shown;
[0054] Figure 5 A structural diagram showing an exemplary hardware architecture of a computing device capable of implementing the method and apparatus for morphology and posture estimation according to an embodiment of the present invention. DETAILED DESCRIPTION
[0055] The features and exemplary embodiments of various aspects of the present invention will be described in detail below. In order to make the purpose, technical solutions and advantages of the present invention clearer, the present invention will be further described in detail below in conjunction with the accompanying drawings and Examples. It should be understood that the specific embodiments described herein are only configured to explain the present invention and are not configured to limit the present invention. For those skilled in the art, the present invention can be implemented without the need for some of these specific details. The following description of the embodiments is only to provide a better understanding of the present invention by illustrating examples of the present invention.
[0056] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the statement "include..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.
[0057] In order to solve the problems in the prior art, an embodiment of the present invention provides a morphology and posture estimation method, apparatus, device and storage medium. The morphology and posture estimation method provided by an embodiment of the present invention is first described below.
[0058] like Figure 1 As shown, Figure 1 A schematic flow chart of a morphology and posture estimation method provided by an embodiment of the present invention is shown. The morphology and posture estimation method may include:
[0059] S101: Collecting three-dimensional surface morphological information of a target object using a three-dimensional camera.
[0060] The target object includes a target organ or a target biological tissue.
[0061] S102: Acquire an initial three-dimensional model corresponding to the target object.
[0062] S103: Matching the surface three-dimensional morphological information and the initial three-dimensional model using a preset matching function to estimate the shape and posture of the target object.
[0063] For organs and biological tissues, the initial three-dimensional model corresponding to the organs or biological tissues can be established by measuring instruments such as computed tomography (CT), magnetic resonance imaging (MRI) and positron emission computed tomography (PET).
[0064] When estimating the shape and posture of an organ or biological tissue, it is possible to directly know what kind of organ or biological tissue the organ or biological tissue to be tested is, such as the heart, lungs, and kidneys, and then directly obtain the corresponding initial three-dimensional model.
[0065] In one embodiment of the present invention, the expression corresponding to the preset matching function is:
[0066] (V',x,y,z,α,β,θ)=H(V,So) (1)
[0067] Wherein, in expression (1), V' is the estimated shape of the target object, (x, y, z) is the coordinate information included in the estimated posture of the target object, (α, β, θ) is the rotation pointing angle included in the estimated posture of the target object, V is the initial three-dimensional model corresponding to the target object, So is the surface three-dimensional shape information of the target object, and H is the corresponding rule of the preset matching function, which aims to minimize the difference between So and V'.
[0068] In one embodiment of the present invention, for the surface three-dimensional morphology information So and the initial three-dimensional model V, multiple groups of corresponding estimated forms V', coordinate information (x, y, z) and rotation pointing angles (α, β, θ) can be obtained, and each group of estimated forms V' is compared with the surface three-dimensional morphology information So to obtain the estimated form V' with the smallest difference from the surface three-dimensional morphology information So. The estimated form V' with the smallest difference from the surface three-dimensional morphology information So is used as the final estimated form, and the coordinate information (x, y, z) and rotation pointing angle (α, β, θ) corresponding to the estimated form are used as the final coordinate information (x, y, z) and rotation pointing angle (α, β, θ) respectively.
[0069] In one embodiment of the present invention, the above-mentioned coordinate information (x, y, z) and rotation pointing angle (α, β, θ) can be adjusted until the difference between the surface three-dimensional morphology information So and the estimated morphology V' is minimized, and the estimated morphology V' at this time is used as the final estimated morphology, and the coordinate information (x, y, z) and rotation pointing angle (α, β, θ) at this time are used as the final coordinate information (x, y, z) and rotation pointing angle (α, β, θ).
[0070] The morphology and posture estimation method of the embodiment of the present invention can utilize the surface three-dimensional morphology information of the organ or biological tissue to estimate the morphology and posture of the organ or biological tissue. It is not based on limited feature points or partial outer contours and is therefore not affected by deformation, occlusion, etc. of the organ or biological tissue, thereby improving the estimation accuracy.
[0071] Whether in open surgery or endoscopic surgery, if doctors can obtain real-time internal structural information of the organs or biological tissues of the current surgical object, such as the location and thickness of blood vessels, it will greatly reduce the doctor's pressure and the risk of surgery. In current medical methods, equipment such as CT, MRI or PET can provide very subtle three-dimensional structures inside human organs and biological tissues. However, since the image display method scanned by traditional CT, MRI and PET equipment and the position and morphology of organs or biological tissues when collecting images are not completely consistent with the surgical process, there are often obstacles when the collected information is applied during the operation. People have been trying to make better use of this three-dimensional information during the operation. The methods currently being tried include virtual three-dimensional preoperative planning, preoperative planning based on 3D printing, intraoperative CT or MRI, and auxiliary information display using augmented reality glasses.
[0072] These existing methods still have great limitations in surgery. First, virtual three-dimensional preoperative planning and 3D printing preoperative planning cannot overlap the three-dimensional internal structure with the patient's actual organs or biological tissues in the field of view. This requires doctors to match the parts seen in front of them during the operation with the three-dimensional model during planning according to their own experience. At the same time, neither of these two methods can provide real-time information guidance for doctors' operations. At present, based on the application of virtual reality (VR), mixed reality (MR) or PET, to some extent, the corresponding organs or biological tissues of the patient in the field of view can be found and a simple superposition can be made. However, this superposition has great limitations. Especially in endoscopic surgery, the field of view of sensors such as cameras is often partially blocked by doctors or other instruments. Due to the size of the space during the operation and the occlusion of other body parts, the field of view of the camera can often only cover part of the organ or biological tissue. During the operation, organs or biological tissues, especially soft organs or biological tissues, often produce morphological deformations that are different from those during CT, MRI or PET scanning due to changes in force. Operations such as organ or biological tissue dissection and partial resection during surgery will also cause significant changes to the appearance of organs or biological tissues. The above points will not only make it difficult for organs or biological tissues to match 3D graphics acquired by CT, MRI or PET, but also make the three-dimensional internal structure acquired by CT, MRI or PET no longer accurately describe the real-time internal structure of organs or biological tissues during surgery, and even make it difficult to correct the model in time when the organ or biological tissue is displaced. At the same time, VR / MR glasses are limited by their display methods and it is difficult to accurately match and display the internal structure of organs or biological tissues. The current method of modeling organs or biological tissues to adapt to the deformation of organs or biological tissues often requires doctors to keep adjusting the model manually during surgery. This greatly increases the burden on doctors during surgery and places high demands on doctors' experience. Due to the limitations of the empirical model itself, the matching accuracy of the model when a specific organ or biological tissue is deformed is also questionable. Intraoperative CT or MRI can re-acquire the current state of the organ or biological tissue, but this only re-acquires the structure affecting the organ or biological tissue, and still cannot achieve real-time update during the entire operation. At the same time, intraoperative CT or MRI brings huge costs and requires huge facilities. At the same time, bringing huge imaging equipment into the operating room and the imaging process itself will also cause many interferences to the surgery.
[0073] Based on this, the morphology and posture estimation method provided by the embodiment of the present invention may further include: estimating the internal structure deformation of the target object according to the surface three-dimensional morphology information.
[0074] Figure 2 FIG. 2 is a schematic diagram showing a second flow chart of the method for estimating the shape and posture provided by an embodiment of the present invention. Figure 2 The embodiment shown and the present invention Figure 1 The difference of the illustrated embodiment is that S104 is added: estimating the internal structural deformation of the target object according to the surface three-dimensional morphology information.
[0075] In one embodiment of the present invention, the internal structural deformation of the target object can be estimated based on the surface three-dimensional morphological information using a neural network model.
[0076] The embodiments of the present invention do not limit the neural network model used, and any available neural network model can be applied to the embodiments of the present invention.
[0077] The neural network model requires a large number of training samples. The training samples can be obtained from animal experiments. In animal experiments, after contrast agents are injected, the organs or biological tissues of the animals are deformed externally and scanned with CT to obtain the structural changes inside the organs or biological tissues. In this way, training samples of internal structural changes caused by external deformation of organs or biological tissues are obtained. In the training process of the neural network model, the experimental data of animal samples and the real CT data of patients are used to train the neural network model with the real CT data of patients as a guide.
[0078] Since data collection of animal samples is difficult, the transfer learning method can be used during the training of neural network models to reduce the data requirements.
[0079] When training a neural network model, the most important part is the collection of training samples. In the medical industry, it is quite difficult to collect the real-time structure inside organs or biological tissues. In order to further reduce the requirements for the size of the training data set after adopting transfer learning, it is necessary to effectively reduce the number of parameters required for the neural network model. In one embodiment of the present invention, finite element modeling can be performed on the structural characteristics of the organ or biological tissue. By finite element modeling of the organ or biological tissue, the expression of the organ or biological tissue is transformed from a three-dimensional lattice to a physical model controlled by a finite number of parameters. In this way, the neural network model is a finite element-based neural network model, and the output of the neural network model is converted from the three-dimensional lattice of the directly predicted organ or biological tissue to the parameters of the output predicted physical model of the organ or biological tissue. Therefore, this method uses the internal structural characteristics of the organ or biological tissue to assist the deep learning network, thereby reducing the difficulty of the deep learning network to predict accurate results, thereby reducing the requirements for the size of the training sample set.
[0080] In one embodiment of the present invention, when estimating the internal structural deformation of a target object based on the surface three-dimensional morphology information, the change range of the target object surface can be determined based on the surface three-dimensional morphology information acquired twice adjacently; and the internal structural deformation of the target object can be estimated based on the change range.
[0081] The embodiment of the present invention does not limit the method of determining the change range of the surface of the target object based on the surface three-dimensional morphology information obtained twice adjacently, and any available method can be applied to the embodiment of the present invention.
[0082] In one embodiment of the present invention, if the change amplitude is smaller than a preset first threshold, it is estimated that the internal structure of the target object has not been deformed.
[0083] In one embodiment of the present invention, if the variation amplitude is smaller than a preset first threshold, the surface deformation of the target object may be directly applied to the internal structure of the target object.
[0084] In one embodiment of the present invention, if the change amplitude is greater than a preset second threshold, the change volume corresponding to the surface change of the target object is calculated based on the surface three-dimensional morphological information acquired twice adjacently; and the change volume is determined as the internal structure deformation of the target object.
[0085] The embodiment of the present invention does not limit the method of calculating the change volume corresponding to the surface change of the target object based on the surface three-dimensional morphological information obtained twice adjacently, and any available method can be applied to the embodiment of the present invention.
[0086] Slight deformations are often absorbed by the surface of the organ or biological tissue itself, and will not be transmitted to the inside of the organ or biological tissue. In order to deal with this situation and improve the accuracy of predicting internal structural deformations, subtle changes in surface morphology can be pre-filtered. For example, based on the actual characteristics of organs or biological tissues, small deformations in a certain area can be ignored. In a specific implementation plan, the surface deformation area and threshold can be manually divided according to the model. It can also be automatically divided through deep learning methods.
[0087] In one embodiment of the present invention, the estimated shape V' obtained by the above-mentioned preset matching function can well fit the surface three-dimensional shape information So observed by the three-dimensional camera in the actual scene, but the accuracy of its estimation of the structural parts that cannot be directly observed inside the organs or biological tissues will be reduced to a certain extent.
[0088] For organs or biological tissues, the principles used in the image registration methods commonly used in CT, MRI or PET images can be used to fine-tune the deformed internal structure. These image registration methods include but are not limited to: harmonic energy, surface conformal structural distortion, minimizing flow field energy, optimal transmission methods, etc.
[0089] Furthermore, the accuracy of the estimation of the internal structure of an organ or biological tissue that cannot be directly observed can be corrected by deep learning methods. Let the transformation function of the deep learning neural network be D, and the internal structure of the internal structure of the organ or biological tissue can be obtained according to expression (2).
[0090] V'=D(V,So') (2)
[0091] In expression (2), V is the initial three-dimensional model of the organ or biological tissue obtained by CT, MRI or PET, and So' is the representation of the three-dimensional morphological information So of the surface of the organ or biological tissue mapped back to the initial three-dimensional model space according to the above preset matching function.
[0092] In one embodiment of the present invention, the three-dimensional model can be displaced and rotated so that a certain surface of the three-dimensional model roughly overlaps with the observed surface; the surface is deformed within a certain permitted range according to the corresponding physical properties such as the hardness of the object; the coordinate difference between the model surface simulating the deformation and the actual observed surface is calculated as a measure; the three-dimensional model is rotated at a certain angle so that another part of the surface roughly overlaps with the observed surface; the steps of deforming the surface within a certain permitted range according to the corresponding physical properties such as the hardness of the object; calculating the coordinate difference between the model surface simulating the deformation and the actual observed surface as a measure; rotating the three-dimensional model at a certain angle so that another part of the surface roughly overlaps with the observed surface are continued until all surfaces are traversed; the coordinate posture with the smallest corresponding difference is selected as the coordinate posture of the model at this time; and the deformation information of the model surface is transmitted to the structure inside the model according to the physical properties of the model itself.
[0093] The morphology and posture estimation method of the embodiment of the present invention can estimate the internal structural deformation of an organ or biological tissue.
[0094] Corresponding to the above method embodiment, the embodiment of the present invention also provides a form and posture estimation device. Figure 3 As shown, Figure 3 The first structural schematic diagram of the morphology and posture estimation device provided by the embodiment of the present invention is shown. The morphology and posture estimation device may include:
[0095] The acquisition module 301 is used to acquire the three-dimensional surface morphological information of the target object using a three-dimensional camera.
[0096] The target object includes a target organ or a target biological tissue.
[0097] The acquisition module 302 is used to acquire an initial three-dimensional model corresponding to the target object.
[0098] The estimation module 303 is used to match the surface three-dimensional morphological information with the initial three-dimensional model using a preset matching function, and estimate the morphology and posture of the target object.
[0099] In one embodiment of the present invention, the expression corresponding to the preset matching function is:
[0100] (V',x,y,z,α,β,θ)=H(V,So) (3)
[0101] Among them, in expression (3), V' is the estimated shape of the target object, (x, y, z) is the coordinate information included in the estimated posture of the target object, (α, β, θ) is the rotation pointing angle included in the estimated posture of the target object, V is the initial three-dimensional model corresponding to the target object, So is the surface three-dimensional shape information of the target object, and H is the corresponding rule of the preset matching function, and the preset matching function aims to minimize the difference between So and V'.
[0102] The organ or biological tissue morphology and posture estimation device of the embodiment of the present invention can use the surface three-dimensional morphology information of the organ or biological tissue to estimate the morphology and posture of the organ or biological tissue. It is not based on limited feature points or partial outer contours and is therefore not affected by deformation, occlusion, etc. of the organ or biological tissue, thereby improving the estimation accuracy.
[0103] Figure 4 FIG. 2 shows a second structural diagram of a device for estimating a form and posture provided by an embodiment of the present invention. Figure 4 The embodiment shown and the present invention Figure 3 The difference of the illustrated embodiment is that an internal structure deformation estimation module 304 is added.
[0104] The internal structure deformation estimation module 304 is used to estimate the internal structure deformation of the target object according to the surface three-dimensional morphology information.
[0105] In one embodiment of the present invention, the internal structure deformation estimation module 304 may be specifically used for:
[0106] Based on the three-dimensional surface morphology information, the neural network model is used to estimate the internal structure deformation of the target object.
[0107] In one embodiment of the present invention, the neural network model is a finite element based neural network model.
[0108] In one embodiment of the present invention, the internal structure deformation estimation module 304 may include:
[0109] A change range determination unit, used to determine the change range of the surface of the target object according to the three-dimensional surface morphology information acquired twice adjacently;
[0110] The internal structure deformation estimation unit is used to estimate the internal structure deformation of the target object according to the change amplitude.
[0111] In one embodiment of the present invention, the internal structure deformation estimation unit can be specifically used for:
[0112] If the change amplitude is smaller than the preset first threshold, it is estimated that the internal structure of the target object has not been deformed.
[0113] In one embodiment of the present invention, the internal structure deformation estimation unit can be specifically used for:
[0114] If the change amplitude is greater than a preset second threshold, the change volume corresponding to the change of the surface of the target object is calculated according to the surface three-dimensional morphology information acquired twice adjacently;
[0115] The changed volume is determined as the internal structure deformation of the target object.
[0116] The morphology and posture estimation device of the embodiment of the present invention can estimate the internal structural deformation of an organ or biological tissue.
[0117] Figure 5 FIG. 1 is a block diagram showing an exemplary hardware architecture of a computing device capable of implementing the method and apparatus for estimating the form and posture according to an embodiment of the present invention. Figure 5 As shown, the computing device 500 includes an input device 501, an input interface 502, a central processing unit 503, a memory 504, an output interface 505, and an output device 506. The input interface 502, the central processing unit 503, the memory 504, and the output interface 505 are connected to each other via a bus 510, and the input device 501 and the output device 506 are connected to the bus 510 via the input interface 502 and the output interface 505, respectively, and are further connected to other components of the computing device 500.
[0118] Specifically, the input device 501 receives input information from the outside and transmits the input information to the central processing unit 503 through the input interface 502; the central processing unit 503 processes the input information based on the computer executable instructions stored in the memory 504 to generate output information, temporarily or permanently stores the output information in the memory 504, and then transmits the output information to the output device 506 through the output interface 505; the output device 506 outputs the output information to the outside of the computing device 500 for user use.
[0119] That is to say, Figure 5 The computing device shown can also be implemented as a morphology and posture prediction device, which may include: a memory storing a computer program that can be run on a processor; and a processor, which can implement the morphology and posture prediction method and apparatus provided in an embodiment of the present invention when executing the computer program.
[0120] An embodiment of the present invention further provides a computer-readable storage medium having a computer program stored thereon; when the computer program is executed by a processor, the form and posture estimation method provided by the embodiment of the present invention is implemented.
[0121] It should be clear that the present invention is not limited to the specific configuration and processing described above and shown in the figures. For the sake of simplicity, a detailed description of the known method is omitted here. In the above embodiments, several specific steps are described and shown as examples. However, the method process of the present invention is not limited to the specific steps described and shown, and those skilled in the art can make various changes, modifications and additions, or change the order between the steps after understanding the spirit of the present invention.
[0122] The functional blocks shown in the above-described block diagram can be implemented as hardware, software, firmware or a combination thereof. When implemented in hardware, it can be, for example, an electronic circuit, an application specific integrated circuit (ASIC), appropriate firmware, a plug-in, a function card, etc. When implemented in software, the elements of the present invention are programs or code segments that are used to perform the required tasks. The program or code segment can be stored in a machine-readable medium, or transmitted on a transmission medium or a communication link by a data signal carried in a carrier wave. "Machine-readable medium" can include any medium capable of storing or transmitting information. Examples of machine-readable media include electronic circuits, semiconductor memory devices, ROM, flash memory, erasable ROM (EROM), floppy disks, CD-ROMs, optical disks, hard disks, optical fiber media, radio frequency (RF) links, etc. The code segment can be downloaded via a computer network such as the Internet, an intranet, etc.
[0123] It should also be noted that the exemplary embodiments mentioned in the present invention describe some methods or systems based on a series of steps or devices. However, the present invention is not limited to the order of the above steps, that is, the steps can be performed in the order mentioned in the embodiments, or in a different order from the embodiments, or several steps can be performed simultaneously.
[0124] The above is only a specific implementation of the present invention. Those skilled in the art can clearly understand that for the convenience and simplicity of description, the specific working process of the system, module and unit described above can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here. It should be understood that the protection scope of the present invention is not limited to this. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed by the present invention, and these modifications or replacements should be covered within the protection scope of the present invention.
Claims
1. A method for shape and posture estimation, It is characterized in that The method comprises: Using a three-dimensional camera to collect three-dimensional surface morphological information of a target object, wherein the target object includes a target organ or a target biological tissue; Acquire an initial three-dimensional model corresponding to the target object; Using a preset matching function to match the surface three-dimensional morphological information with the initial three-dimensional model, estimating the shape and posture of the target object; the posture includes coordinate information and a rotation pointing angle; The method further comprises: estimating the internal structural deformation of the target object according to the surface three-dimensional morphology information; The estimating the internal structural deformation of the target object according to the surface three-dimensional morphology information includes: Determining a change range of the surface of the target object according to the surface three-dimensional morphology information acquired twice adjacently; The internal structural deformation of the target object is estimated according to the variation range.
2. The method according to claim 1, It is characterized in that The expression corresponding to the preset matching function is: (V',x,y,z,α,β,θ)=H(V,So) Among them, V' is the estimated shape of the target object, (x, y, z) is the coordinate information included in the estimated posture of the target object, (α, β, θ) is the rotation pointing angle included in the estimated posture of the target object, V is the initial three-dimensional model corresponding to the target object, So is the surface three-dimensional morphology information of the target object, H is the corresponding rule of the preset matching function, and the preset matching function aims to minimize the difference between So and V'.
3. The method according to claim 1, It is characterized in that The estimating the internal structural deformation of the target object according to the surface three-dimensional morphology information includes: According to the surface three-dimensional morphological information, a neural network model is used to estimate the internal structural deformation of the target object.
4. The method according to claim 1, It is characterized in that The estimating the internal structural deformation of the target object according to the change amplitude includes: If the change amplitude is smaller than a preset first threshold, it is estimated that the internal structure of the target object has not been deformed.
5. The method according to claim 1, It is characterized in that The estimating the internal structural deformation of the target object according to the change amplitude includes: If the change amplitude is greater than a preset second threshold, calculating the change volume corresponding to the surface change of the target object according to the surface three-dimensional morphology information acquired twice adjacently; The changed volume is determined as the internal structural deformation of the target object.
6. A shape and posture estimation device, It is characterized in that The device comprises: A collection module, used for collecting the surface three-dimensional morphological information of the target object by using a three-dimensional camera, wherein the target object includes a target organ or a target biological tissue; An acquisition module, used to acquire an initial three-dimensional model corresponding to the target object; An estimation module, used to match the surface three-dimensional morphological information with the initial three-dimensional model using a preset matching function, and estimate the morphology and posture of the target object; the posture includes coordinate information and a rotation pointing angle; The device also includes: An internal structure deformation estimation module is used to estimate the internal structure deformation of the target object based on the surface three-dimensional morphological information; The internal structure deformation estimation module includes: a change amplitude determining unit, configured to determine a change amplitude of the surface of the target object according to the surface three-dimensional morphological information acquired twice adjacently; The internal structure deformation estimation unit is used to estimate the internal structure deformation of the target object according to the variation range.
7. A shape and posture estimation device, It is characterized in that The device comprises: a memory, a processor, and a computer program stored in the memory and executable on the processor; When the processor executes the computer program, the form and posture estimation method as described in any one of claims 1 to 5 is implemented.
8. A computer-readable storage medium, It is characterized in that The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the form and posture estimation method according to any one of claims 1 to 5 is implemented.
