Method and apparatus for generating postoperative three-dimensional model of surgical subject

Abstract

The application discloses a method and device for generating a postoperative three-dimensional model of a surgical object. The features include: obtaining a first three-dimensional model of the surgical object through a three-dimensional scanning system, and obtaining a second three-dimensional model of the surgical object through medical imaging technology; determining a surgical correction model of the surgical object based on the second three-dimensional model; and performing correction processing on the first three-dimensional model based on the surgical correction model to obtain a postoperative three-dimensional model of the surgical object. The embodiment of the application realizes accurate and efficient simulation of a postoperative three-dimensional body image of the surgical object, and improves the visualization effect of the postoperative effect.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of image processing, and in particular to a method and device for generating a postoperative three-dimensional model of a surgical object. BACKGROUND

[0002] As a main method of surgical treatment, surgery is performed on a local part of the human body by using medical instruments. Since bone correction surgery usually involves cutting and other operation methods, patients have a certain fear of surgical treatment. In order to eliminate the fear of patients and improve the quality of surgery, simulating the surgical process and simulating the surgical result have become a necessary means before surgery.

[0003] In related technologies, postoperative simulation of surgery is often limited to displaying the postoperative effect of the local area affected by surgery through professional medical images, which is not intuitive and cannot reflect the overall impact of surgery on the patient's body posture, so that the user cannot intuitively understand the postoperative effect. SUMMARY

[0004] The present application provides a method and device for generating a postoperative three-dimensional model of a surgical object, an electronic device and a storage medium, which accurately and efficiently simulate the postoperative three-dimensional body image of a surgical object.

[0005] According to an aspect of the present application, a method for generating a postoperative three-dimensional model of a surgical object is provided, comprising:

[0006] obtaining a first three-dimensional model of the surgical object by a three-dimensional scanning system, and obtaining a second three-dimensional model of the surgical object by a medical imaging technique;

[0007] determining a surgical correction model of the surgical object based on the second three-dimensional model;

[0008] correcting the first three-dimensional model based on the surgical correction model to obtain a postoperative three-dimensional model of the surgical object.

[0009] According to another aspect of the present application, a device for generating a postoperative three-dimensional model of a surgical object is provided, comprising:

[0010] a preoperative model acquisition module configured to obtain a first three-dimensional model of the surgical object by a three-dimensional scanning system, and obtain a second three-dimensional model of the surgical object by a medical imaging technique;

[0011] a surgical correction model determination module configured to determine a surgical correction model of the surgical object based on the second three-dimensional model;

[0012] A postoperative model generation module is configured to correct the first three-dimensional model based on the surgical correction model to obtain a postoperative three-dimensional model of the surgical subject.

[0013] According to another aspect of the present application, an electronic device is provided, which comprises:

[0014] at least one processor; and

[0015] a memory in communication with the at least one processor; wherein

[0016] The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor to enable the at least one processor to perform the method for generating a postoperative three-dimensional model of a surgical subject according to any one of the embodiments of the present application.

[0017] According to another aspect of the present application, a computer readable storage medium is provided, which stores computer instructions for enabling a processor to perform the method for generating a postoperative three-dimensional model of a surgical subject according to any one of the embodiments of the present application.

[0018] The technical solution of the embodiments of the present application obtains a first three-dimensional model of a surgical subject through a three-dimensional scanning system, and obtains a second three-dimensional model of the surgical subject through medical imaging technology. The first three-dimensional model and the second three-dimensional model of the surgical subject are obtained, and the three-dimensional image of the surgical subject is simulated based on the two technologies, so as to realize all-around display of the postoperative condition of the surgical subject. The surgical correction model of the surgical subject is determined based on the second three-dimensional model, and the postoperative effect can be more accurately determined based on the medical image. The first three-dimensional model is corrected based on the surgical correction model to obtain a postoperative three-dimensional model of the surgical subject, the postoperative effect of the second three-dimensional model is transferred to the scanned first three-dimensional model, and accurate simulation of the postoperative three-dimensional body image of the surgical subject is realized. The technical problem that the postoperative effect is not intuitive is solved, the simulation of the postoperative effect is effectively performed, and the visualization effect of the simulation of the postoperative effect is improved.

[0019] It should be understood that the content described in this part is not intended to identify key or important features of the embodiments of the present application, nor to limit the scope of the present application. Other features of the present application will become apparent from the following description. BRIEF DESCRIPTION OF DRAWINGS

[0020] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following will briefly introduce the drawings needed in the embodiment description. Obviously, the drawings in the following description only show some embodiments of the present application, and for those skilled in the art, other drawings can also be obtained without creative labor.

[0021] Figure 1 A flow chart of a method for generating a postoperative three-dimensional model of a surgical object is provided for the embodiments of the present application.

[0022] Figure 2 Another cutting schematic diagram of a three-dimensional model of a bone of a surgical object is provided for the embodiments of the present application.

[0023] Figure 3 Another flow chart of a method for generating a postoperative three-dimensional model of a surgical object is provided for the embodiments of the present application.

[0024] Figure 4 A three-dimensional model reference diagram of a postoperative three-dimensional model is provided for the embodiments of the present application.

[0025] Figure 5 Another three-dimensional model reference diagram of a postoperative three-dimensional model is provided for the embodiments of the present application.

[0026] Figure 6 A structural schematic diagram of a generating device of a postoperative three-dimensional model of a surgical object is provided for the embodiments of the present application.

[0027] Figure 7 A structural schematic diagram of an electronic device that can be used to implement the embodiments of the present application is shown. DETAILED DESCRIPTION

[0028] In order to make the technical personnel in the art better understand the present application scheme, the following will combine the drawings in the embodiments of the present application, and the technical solutions in the embodiments of the present application will be described clearly and completely. Obviously, the described embodiments are only some of the embodiments of the present application, not all. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor should belong to the scope of protection of the present application.

[0029] It is to be understood that the terms "first", "second", and the like used in the description and the claims of the present application as well as the above-described drawings do not necessarily have to connote any ordinal, sequential or priority relationship, and are merely used to differentiate between similar objects. It is to be understood that the terms "comprising", "having", "including" and "containing" and any variations thereof used herein are intended to cover a non-exclusive inclusion, such that a process, method, system, product or apparatus that comprises, has, includes or contains a list of steps or elements, but not only those recited as essential components, can also include other steps or elements not expressly listed or inherent to such process, method, system, product or apparatus.

[0030] Figure 1 A flowchart of a method for generating a postoperative three-dimensional model of a surgical subject is provided for an embodiment of the present application. The embodiment can be applicable to the case of simulating a three-dimensional body image of a surgical subject after surgery. The method can be performed by a postoperative three-dimensional model generation device of a surgical subject. The postoperative three-dimensional model generation device of a surgical subject can be realized in the form of hardware and / or software. The postoperative three-dimensional model generation device of a surgical subject can be configured in an electronic device. As shown in FIG. 1, the method comprises: Figure 1

[0031] S110, acquiring a first three-dimensional model of the surgical subject by a three-dimensional scanning system, and acquiring a second three-dimensional model of the surgical subject by a medical imaging technology.

[0032] The three-dimensional scanning system can be used to scan the shape and texture data of the surgical subject, and then construct a three-dimensional model of the body surface of the surgical subject. The three-dimensional scanning system can include at least one of a laser three-dimensional scanner, an infrared structured light camera, and a non-structured light camera. It should be noted that the three-dimensional scanning system is composed of a plurality of three-dimensional scanning devices with three-dimensional scanning function, which can scan the surgical subject from multiple angles such as face, head, upper body and whole body, and then acquire three-dimensional geometric coordinate data and color texture data of the surgical subject, and then construct the first three-dimensional model. The first three-dimensional model can be a three-dimensional model of the shape and surface of the surgical subject.

[0033] The medical imaging technology can be used to acquire information of the surgical subject's physiological structure, determine the physiological related three-dimensional model of the surgical subject through the surgical subject's physiological structure information, and can include at least one of X-ray imaging technology, magnetic resonance imaging technology, computed tomography technology, nuclear medicine imaging and medical ultrasound image. The second three-dimensional model can be a three-dimensional model for describing the area where the target surgical site of the surgical subject is located.

[0034] ​Specifically, shape and texture data of the surgical object are acquired by a three-dimensional scanning system, and a first three-dimensional model of the surgical object is constructed according to the shape and texture data; information of the physiological structure of the surgical object is acquired by a medical imaging technology, and a second three-dimensional model of the surgical object is constructed.

[0035] In S120, a surgical correction model of the surgical object is determined based on the second three-dimensional model.

[0036] The surgical correction model can be a three-dimensional model obtained by performing surgical correction on the second three-dimensional model.

[0037] Specifically, after the second three-dimensional model of the surgical object is constructed, a position to be corrected of the surgical object is determined according to the second three-dimensional model, and a surgical correction model of the surgical object is determined according to the position to be corrected of the surgical object.

[0038] Optionally, in another optional embodiment of the present application, the surgical correction model includes a bone correction model; the determination of the surgical correction model of the surgical object based on the second three-dimensional model includes: segmenting the second three-dimensional model to obtain a bone three-dimensional model of the surgical object; and performing correction processing on the bone three-dimensional model to obtain the bone correction model of the surgical object.

[0039] The bone three-dimensional model can be a three-dimensional model of the bone of the surgical object segmented from the second three-dimensional model. The bone correction model can be a three-dimensional model simulating the correction effect after performing bone correction processing on the surgical object.

[0040] Optionally, the second three-dimensional model includes physiological tissue information of the surgical object, the bone structure information of the surgical object is acquired in the second three-dimensional model, and a bone three-dimensional model corresponding to the bone structure information is determined.

[0041] Optionally, the bone three-dimensional model of the surgical object is acquired, the surgical position is determined in the bone three-dimensional model, and the surgical position in the bone three-dimensional model is subjected to correction processing to obtain the bone correction model of the surgical object.

[0042] An exemplary diagram for cutting the bone three-dimensional model of the surgical object is provided. Figure 2 Another exemplary diagram for cutting the bone three-dimensional model of the surgical object is provided. Figure 2The bone three-dimensional model is observed from six directions, an osteotomy point is selected as a surgical position in the bone three-dimensional model, two cutting planes are constructed for the osteotomy point, the cutting planes are connected to the osteotomy point, the osteotomy point can be used as a correction rotation point, and the rotation position can be determined by dragging the osteotomy point. The bone between the two cutting planes in the bone three-dimensional model is cut, the bone three-dimensional model is divided into two parts, i.e., an upper bone three-dimensional model and a lower bone three-dimensional model, the upper bone three-dimensional model and the lower bone three-dimensional model are fused and spliced to obtain a bone correction model of the surgical object.

[0043] Specifically, a second three-dimensional model of a surgical object is obtained, the second three-dimensional model is segmented to obtain a bone three-dimensional model of the surgical object, a surgical position is determined in the bone three-dimensional model, and the surgical position in the bone three-dimensional model is corrected to obtain a bone correction model of the surgical object.

[0044] To ensure that the effect of bone correction meets the requirements of surgery, the effect of bone correction can be determined by three-dimensional finite element analysis. Then, the bone correction model that meets the preset requirements is used as the bone correction model. Optionally, in another optional embodiment of the present application, the bone three-dimensional model is corrected to obtain the bone correction model of the surgical object, including: the bone three-dimensional model is corrected to obtain a preliminary correction model of the surgical object; the preliminary correction model is analyzed by three-dimensional finite element analysis to determine the correction effect of the preliminary correction model; if the correction effect meets the preset condition, the preliminary correction model is used as the bone correction model of the surgical object.

[0045] The preliminary correction model can be used for the preliminary correction of the bone three-dimensional model. The three-dimensional finite element analysis can be a simulation of the preliminary correction model, which is used to analyze the correction reliability of the preliminary correction model. The preset condition can be that the maximum stress of the preliminary correction model is less than a preset stress threshold.

[0046] Specifically, a bone three-dimensional model of a surgical object is obtained, a surgical position in the bone three-dimensional model is determined, the surgical position in the bone three-dimensional model is corrected to obtain a preliminary correction model of the surgical object, and then the preliminary correction model is analyzed by three-dimensional finite element analysis to obtain a three-dimensional finite element model, the stress distribution in the three-dimensional finite element model is calculated, the correction effect of the preliminary correction model is determined, if the correction effect meets the preset condition, the preliminary correction model is used as the bone correction model of the surgical object; if the correction effect does not meet the preset condition, the bone three-dimensional model is re-corrected.

[0047] S130, the first three-dimensional model is corrected based on the surgical correction model to obtain a postoperative three-dimensional model of the surgical object.

[0048] The postoperative three-dimensional model can be used to simulate the postoperative physical appearance of the surgical subject.

[0049] Specifically, after determining the surgical correction model, the first three-dimensional model is corrected by the surgical correction model, the postoperative correction effect of the second three-dimensional model is mapped to the first three-dimensional model, and the postoperative three-dimensional model of the surgical subject is obtained, which is used to simulate the postoperative physical appearance of the surgical subject.

[0050] The technical scheme of the embodiment of the present application obtains the first three-dimensional model of the surgical subject through a three-dimensional scanning system, and obtains the second three-dimensional model of the surgical subject through medical imaging technology, obtains the first three-dimensional model of the human body and the second three-dimensional model of the medical treatment for the surgical subject, simulates the three-dimensional image of the surgical subject based on the two technologies, realizes the all-around display of the postoperative condition of the surgical subject, determines the surgical correction model of the surgical subject based on the second three-dimensional model, can more accurately determine the postoperative effect based on the medical image, corrects the first three-dimensional model based on the surgical correction model, obtains the postoperative three-dimensional model of the surgical subject, transfers the postoperative effect of the second three-dimensional model to the scanned first three-dimensional model, and realizes the accurate simulation of the postoperative three-dimensional body image of the surgical subject. The technical problem of the postoperative effect not being intuitive is solved, the postoperative effect is effectively simulated, and the visualization effect of the postoperative effect simulation is improved.

[0051] Figure 3 The flowchart of another method for generating a postoperative three-dimensional model of a surgical subject provided by the embodiment of the present application is described in the above embodiment based on the surgical correction model for correcting the first three-dimensional model, and the method of correction is further described. As shown in Figure 3 The generation of the postoperative three-dimensional model of the surgical subject includes:

[0052] S310, obtaining the first three-dimensional model of the surgical subject through a three-dimensional scanning system, and obtaining the second three-dimensional model of the surgical subject through medical imaging technology.

[0053] S320, determining the surgical correction model of the surgical subject based on the second three-dimensional model.

[0054] S330, registering the first three-dimensional model and the second three-dimensional model to determine the model registration information between the first three-dimensional model and the second three-dimensional model.

[0055] The model registration information can be used for rectification of the first three-dimensional model. It should be noted that the second three-dimensional model includes physiological tissue information of the surgical object. In general, the second three-dimensional model contains part of the physiological tissue information of the surgical object. The first three-dimensional model and the second three-dimensional model need to be registered. The model registration information can be a geometric fitting model between the first three-dimensional model and the second three-dimensional model, and is used for rectification of the first three-dimensional model.

[0056] Specifically, the first three-dimensional model and the second three-dimensional model are obtained, the first three-dimensional model and the second three-dimensional model are registered, and the model registration information between the first three-dimensional model and the second three-dimensional model is determined.

[0057] Optionally, in another optional embodiment of the present application, the registration of the first three-dimensional model and the second three-dimensional model includes: obtaining a plurality of first marking points in the first three-dimensional model and second marking points corresponding to the first marking points in the second three-dimensional model; determining the model registration information between the first three-dimensional model and the second three-dimensional model based on the first marking points, the second marking points, and a preset model registration algorithm, wherein the model registration algorithm includes at least one of affine transformation, perspective transformation, and polynomial transformation.

[0058] The first marking points can be in the first three-dimensional model and are used for registration. The second marking points can be in the second three-dimensional model and are used for registration. It should be noted that when the three-dimensional scanning system obtains the surface information of the surgical object, the marking points are set on the surgical object in advance, so that the marking points of the first three-dimensional model are determined according to the marking points of the surgical object when the first three-dimensional model is established. When the physiological tissue information of the surgical object is obtained by medical imaging technology, the marking points are set on the surgical object, so that the second marking points are determined according to the marking points of the surgical object when the second three-dimensional model is established.

[0059] The model registration algorithm can be a pre-set algorithm for configuration.

[0060] Specifically, a plurality of first marking points in the first three-dimensional model and second marking points corresponding to the first marking points in the second three-dimensional model are obtained. The first marking points and the second marking points are correspondingly matched. The first marking points and the second marking points are rigidly registered according to a pre-set model matching algorithm. The model registration information between the first three-dimensional model and the second three-dimensional model is determined. The model registration algorithm includes at least one of affine transformation, perspective transformation, and polynomial transformation, but is not limited to these transformations. Typically, affine transformation can be used.

[0061] Optionally, in another optional embodiment of the present application, after the model registration information between the first three-dimensional model and the second three-dimensional model is determined, the method further comprises: updating the model registration information between the first three-dimensional model and the second three-dimensional model based on an iterative nearest neighbor point algorithm.

[0062] The iterative nearest neighbor point algorithm can be used to update the model registration information of the first marker points and the second marker points. It should be noted that the iterative nearest neighbor point algorithm at least includes a distance term loss function, a rigidity term loss function and a key point term. The distance term can calculate the distance between each first marker point and the corresponding second marker point and sum them up, and the rigidity term can constrain the transformation of adjacent marker points. The key point term can be used to guide the registration so that the first marker points and the second marker points are as close as possible in the registration process.

[0063] Optionally, the first marker points and the second marker points are registered to obtain model registration information. The distance between each first marker point and the corresponding second marker point is calculated and summed up by the iterative nearest neighbor point algorithm to obtain a distance sum value. It is judged whether the distance sum value is greater than a preset threshold. If the distance sum value is greater than the preset threshold, the iterative nearest neighbor point algorithm is used for optimization. The corresponding model information of the first three-dimensional model and the second three-dimensional model is calculated based on the corresponding relationship between the first marker points and the second marker points. The model information is cropped to obtain target information. The model registration information is updated based on the target information and the second three-dimensional model. The preset threshold can be a threshold preset for judging the effect of rigid registration.

[0064] S340, based on the surgical correction model and the model registration information, performing correction processing on the first three-dimensional model to obtain a postoperative three-dimensional model of the surgical object.

[0065] Optionally, a first reference point in the surgical correction model is determined. First position transformation information of the first reference point in the second three-dimensional model and the surgical correction model is determined. Second position transformation information corresponding to each surface pixel point in the first three-dimensional model is determined based on the model registration information and the first position transformation information. Each surface pixel point is subjected to position transformation based on the second position transformation information corresponding to each surface pixel point.

[0066] The first reference point can be a reference point for position change of the surgical correction model. It should be noted that the morphology of the physiological tissue information in the surgical correction model will change when the surgical correction model is subjected to correction processing. At this time, the point with the largest change amplitude in the surgical correction model can be selected as the first reference point. For example, when the surgical object is subjected to an osteotomy surgery, the first reference point in the surgical correction model can be the osteotomy point of the surgical object.

[0067] The first position change information can be used to change the position of the first reference point in the second three-dimensional model relative to the position in the surgical correction model. The surface pixel point can be a pixel point of a texture surface in the first three-dimensional model. The second position change information can be used to change the position of the surface pixel point in the first three-dimensional model to the change information that occurs at the position corresponding to the position of the first reference point in the surgical correction model.

[0068] Optionally, when the first three-dimensional model is corrected by the surgical correction model, the first reference point in the surgical correction model can be obtained first, and then the first position change information of the first reference point in the second three-dimensional model and the surgical correction model is determined according to the first reference point. Then, each surface pixel point in the first three-dimensional model that needs to be changed in position is obtained, the second position change information corresponding to each surface pixel point in the first three-dimensional model is determined through the model registration information and the first position change information, and finally, each surface pixel point is changed in position based on the second position change information corresponding to each surface pixel point. The advantage of this is that the correction part can be smoothly simulated, the correction part can be avoided to have defects, and the three-dimensional model can be made more realistic.

[0069] Considering that the distances between each surface pixel point and the surgical point are different, the postoperative change amplitudes are also different. Optionally, in another optional embodiment of the present application, the second position change information corresponding to each surface pixel point in the first three-dimensional model is determined based on the model registration information and the first position change information, including: determining a second reference point in the first three-dimensional model corresponding to the first reference point based on the model registration information; for each surface pixel point in the first three-dimensional model, determining distance information between the surface pixel point and the second reference point; and determining the second position change information corresponding to the surface pixel point based on the distance information and the first position change information.

[0070] The second reference point can be a reference point in the first three-dimensional model corresponding to the first reference point, and the distance information can be the distance between each surface pixel point in the first three-dimensional model and the second reference.

[0071] Specifically, the first reference point in the surgical correction model is obtained, the second reference point in the first three-dimensional model corresponding to the first reference point is determined according to the model configuration information, the surface pixel point of the first three-dimensional model is obtained, the distance information between each surface pixel point and the second reference point is determined, and the second position change information corresponding to each surface pixel point is determined according to the distance information and the first position change information.

[0072] Optionally, in another optional embodiment of the present application, the determining the second position transformation information corresponding to the surface pixel point based on the distance information and the first position transformation information comprises: determining a weight corresponding to the surface pixel point based on the distance information, wherein the weight and the distance information are in a positive correlation relationship; and determining the second position transformation information corresponding to the surface pixel point based on the weight and the first position transformation information.

[0073] The weight can be used to calculate the second position transformation information of the surface pixel point. It should be noted that when the surgical correction model is corrected, the change amplitude between each surface pixel point and the second reference point is positively correlated with the distance. When the distance between the surface pixel point and the second reference point is small, the change amplitude is relatively small, and when the distance between each surface pixel point and the second reference point is large, the change amplitude is relatively large. Therefore, the weight and the distance information are in a positive correlation relationship.

[0074] Specifically, the distance information between the surface pixel point and the second reference point is obtained, the weight of the surface pixel point is determined through the positive correlation relationship between the weight and the distance information, and the second position transformation information corresponding to the surface pixel point is determined based on the weight and the first position transformation information.

[0075] Exemplarily, Figure 4 A three-dimensional model reference diagram of a postoperative three-dimensional model is provided for an embodiment of the present application. Figure 5 Another three-dimensional model reference diagram of a postoperative three-dimensional model is provided for an embodiment of the present application. As shown in Figure 4 and Figure 5 As shown in the drawings, after osteotomy is performed on a surgical object, the osteotomy point in the surgical correction model is taken as a first reference point, the first position change relationship of the osteotomy point before and after osteotomy in the second three-dimensional model is determined, the second reference point corresponding to the osteotomy point in the first three-dimensional model is determined through model registration information, the surface pixel points that need to be changed in position after osteotomy in the first three-dimensional model are obtained, the distance information between the surface pixel points and the osteotomy point is calculated, the weight corresponding to the surface pixel points is obtained according to the positive correlation relationship between the distance information and the weight, the second position transformation information corresponding to the surface pixel points is determined according to the weight and the first position change information, and each surface pixel point is subjected to position transformation based on the second position transformation information corresponding to each surface pixel point. As shown in the drawings, Figure 4 In the area corresponding to the osteotomy point, a blurred transition area can be obviously seen; as shown in the drawings, Figure 5 In the area corresponding to the osteotomy point, a smooth transition can be obviously seen, which is more in line with the human body. The formula for subjecting each surface pixel point to position transformation based on the second position transformation information corresponding to each surface pixel point is as follows:

[0076]

[0077] wherein T(x) is a model after position change; N is the number of surface pixel points; i is the i-th surface pixel point; w i (X) is a weight; T i (x) is first position change information.

[0078] The technical scheme of the embodiment of the application determines the model registration information between the first three-dimensional model and the second three-dimensional model by registering the first three-dimensional model and the second three-dimensional model, performs corresponding registration on the first three-dimensional model and the second three-dimensional model, and can improve the accuracy of three-dimensional model simulation. The postoperative three-dimensional model of the surgical object is obtained by performing correction processing on the first three-dimensional model based on the surgical correction model and the model registration information, the postoperative effect of the first three-dimensional model can be more accurately mapped into the second three-dimensional model, and the visual presentation effect of the postoperative correction model is effectively ensured.

[0079] As an optional example of the method for generating the postoperative three-dimensional model of the surgical object disclosed in the embodiment of the application. The method can specifically include:

[0080] S1, human three-dimensional scanning. An infrared structured light camera is installed around the scanning room, a speckle pattern is projected onto the surface of the human body, a plurality of infrared structured light cameras are controlled in linkage, image acquisition of all cameras is quickly completed, three-dimensional scanning of the face, head, upper body and whole body of the human body is realized, three-dimensional geometric coordinate data and color texture data of the human body are obtained, and then a first three-dimensional model is quickly reconstructed from the acquired color image.

[0081] S2, extracting the skin surface from the second three-dimensional model. The second three-dimensional model is obtained by medical imaging technology, the skin surface is extracted from the second three-dimensional model by threshold segmentation, the basic threshold of the skin tissue is set, and finally the human skin surface is drawn by a ray casting algorithm.

[0082] S3, matching the skin surface in the second three-dimensional model and the texture data of the first three-dimensional model. The first marker points and the second marker points in the first three-dimensional model and the second three-dimensional model are matched respectively, according to the geometric distortion condition between the first three-dimensional model and the second three-dimensional model, a geometric transformation model capable of best fitting the changes between the two images is selected, then the first three-dimensional model is subjected to corresponding parameter transformation, so that the first three-dimensional model and the second three-dimensional model are in the same coordinate system. Since the coordinate points after image transformation are not necessarily integers, a certain interpolation processing operation needs to be considered, and the interpolation processing operation includes at least one of nearest neighbor interpolation, bilinear interpolation, bicubic interpolation, B-spline interpolation and Gaussian interpolation.

[0083] The nearest neighbor loss function value is calculated by an iterative nearest neighbor point algorithm, and the nearest neighbor loss function value is obtained by summing the distances of all matched marker point pairs. When the nearest neighbor loss function value is greater than a preset threshold, a non-rigid ICP registration is used. The skin surface of the second three-dimensional model is taken as original data. According to the correspondence between the marker points, the texture surface of the first three-dimensional model corresponding to the upper half of the human body is calculated and cut out as target data. By determining the corresponding model registration parameters for each vertex, the original data can represent the target data. The formula of the non-rigid ICP registration is as follows:

[0084] E = E d (x) + 0.2 * E s (x) + 0.5 * E l (x)

[0085] E = E d (x) is a distance term; E s (x) is a rigid term; E l (x) is a key point term; and E is a model registration parameter.

[0086] S4, three-dimensional preoperative simulation operation. The bone three-dimensional model is obtained by segmenting the second three-dimensional model. The bone three-dimensional model can be freely dragged in six directions for convenient observation and operation. A point is selected on the spine model in the three-dimensional space to represent the osteotomy point, and two planes that can rotate freely in six directions are connected to the osteotomy point as cutting planes. The bone between the two cutting planes is removed, and the entire bone three-dimensional model is divided into two parts. The three-dimensional models of the upper and lower parts can be freely dragged in six directions and spliced to form a preliminary correction model. Various parameters of the spine and pelvis are measured and displayed.

[0087] S5, three-dimensional finite element analysis of the preliminary correction model. Taking a patient with ankylosing spondylitis kyphosis as an example, first, the computer-aided design software is used to optimize the surface of the preliminary correction model and eliminate sharp edges, so that the model is beneficial to the later finite element modeling. Then, the spine solid model is established, and part of the useless sacral solid model is cut off. A load plane is established at the top of the first cervical vertebra to facilitate the setting of the stress condition. The calcified ligament model connecting the vertebral bodies is established using curves and sketches. Then, the established three-dimensional model is imported into the finite element software for finite element meshing, and the intervertebral disc and ligament structure are added. The intervertebral disc and ligament of the patient with ankylosing spondylitis kyphosis are severely ossified, so the same material parameters as the cortical bone are used, and screws and titanium rods are installed at the splicing fusion site. Finally, a complete three-dimensional finite element model is generated. The stress distribution of the vertebral body fixed by the pedicle screw in the calculation model is determined to determine the correction effect of the preliminary correction model. If the correction effect meets the preset condition, the preliminary correction model is taken as the bone correction model of the surgical object.

[0088] S6, the osteotomy point in the surgical correction model is taken as a first reference point, a first position change relationship of the osteotomy point before and after osteotomy in a second three-dimensional model is determined, a second reference point corresponding to the osteotomy point in the first three-dimensional model is determined through model registration information, surface pixel points needing position change after osteotomy are acquired in the first three-dimensional model, distance information between the surface pixel points and the osteotomy point is calculated, a weight corresponding to the surface pixel points is acquired according to a positive correlation relationship between the distance information and the weight, second position transformation information corresponding to the surface pixel points is determined according to the weight and the first position change information, and position transformation is performed on each surface pixel point based on the second position transformation information corresponding to each surface pixel point.

[0089] The technical scheme of the embodiment of the application can solve the technical problem that the influence change of the surgery on the whole patient and the influence change of the patient's body surface cannot be reflected in the prior art, and the visualized presentation of the postoperative effect is realized.

[0090] Figure 6 A structural schematic diagram of a postoperative three-dimensional model generation device of a surgical object is provided in the embodiment of the application. Figure 6 As shown in the figure, the device comprises: a preoperative model acquisition module 610, a surgical correction model determination module 620 and a postoperative model generation module 630.

[0091] The preoperative model acquisition module 610 is configured to acquire a first three-dimensional model of the surgical object through a three-dimensional scanning system and acquire a second three-dimensional model of the surgical object through a medical imaging technology; the surgical correction model determination module 620 is configured to determine a surgical correction model of the surgical object based on the second three-dimensional model; and the postoperative model generation module 630 is configured to perform correction processing on the first three-dimensional model based on the surgical correction model to obtain a postoperative three-dimensional model of the surgical object.

[0092] The technical scheme of the embodiment of the present application obtains the first three-dimensional model of the surgical object through a three-dimensional scanning system, and obtains the second three-dimensional model of the surgical object through a medical imaging technology, obtains the first three-dimensional model of the human body and the second three-dimensional model of the medical object for the surgical object, simulates the three-dimensional image of the surgical object based on two technologies, realizes all-around display of the postoperative condition of the surgical object; determines the surgical correction model of the surgical object based on the second three-dimensional model, and can more accurately determine the postoperative effect based on the medical image; corrects the first three-dimensional model based on the surgical correction model, obtains the postoperative three-dimensional model of the surgical object, transfers the postoperative effect of the second three-dimensional model to the scanned first three-dimensional model, and realizes accurate simulation of the postoperative three-dimensional body image of the surgical object. The technical problem that the postoperative effect is not intuitive is solved, the postoperative effect is effectively simulated, and the visualization effect of the postoperative effect simulation is improved.

[0093] Optionally, the postoperative model generation module is specifically used for:

[0094] Registering the first three-dimensional model and the second three-dimensional model to determine model registration information between the first three-dimensional model and the second three-dimensional model;

[0095] Correcting the first three-dimensional model based on the surgical correction model and the model registration information.

[0096] Optionally, the postoperative model generation module is specifically further used for:

[0097] Obtaining a plurality of first marker points in the first three-dimensional model and second marker points corresponding to the first marker points in the second three-dimensional model;

[0098] Determining model registration information between the first three-dimensional model and the second three-dimensional model based on the first marker points, the second marker points, and a preset model registration algorithm, wherein the model registration algorithm includes at least one of affine transformation, perspective transformation, and polynomial transformation.

[0099] Optionally, the postoperative model generation module is specifically further used for:

[0100] Updating the model registration information between the first three-dimensional model and the second three-dimensional model based on an iterative nearest neighbor point algorithm.

[0101] Optionally, the postoperative model generation module is specifically further used for:

[0102] Determining a first reference point in the surgical correction model, and determining first position transformation information of the first reference point in the second three-dimensional model and the surgical correction model;

[0103] determine second position transformation information corresponding to each surface pixel point in the first three-dimensional model based on the model registration information and the first position transformation information;

[0104] perform position transformation on each surface pixel point based on second position transformation information corresponding to the surface pixel point.

[0105] Optionally, the postoperative model generation module is specifically configured to:

[0106] determine a second reference point in the first three-dimensional model corresponding to the first reference point based on the model registration information;

[0107] for each surface pixel point in the first three-dimensional model, determine distance information between the surface pixel point and the second reference point;

[0108] determine second position transformation information corresponding to the surface pixel point based on the distance information and the first position transformation information.

[0109] Optionally, the postoperative model generation module is specifically configured to:

[0110] determine a weight corresponding to the surface pixel point based on the distance information, wherein the weight is in a positive correlation with the distance information;

[0111] determine second position transformation information corresponding to the surface pixel point based on the weight and the first position transformation information.

[0112] Optionally, the surgical correction model includes a bone correction model; and the surgical correction model determination module is specifically configured to:

[0113] segment the second three-dimensional model to obtain a bone three-dimensional model of the surgical subject;

[0114] perform correction processing on the bone three-dimensional model to obtain a bone correction model of the surgical subject.

[0115] Optionally, the surgical correction model determination module is specifically configured to:

[0116] perform correction processing on the bone three-dimensional model to obtain a preliminary correction model of the surgical subject;

[0117] perform three-dimensional finite element analysis on the preliminary correction model to determine a correction effect of the preliminary correction model;

[0118] if the correction effect meets a preset condition, the preliminary correction model is taken as the bone correction model of the surgical subject.

[0119] The postoperative three-dimensional model device of the surgical object provided by the embodiments of the present disclosure can execute the postoperative three-dimensional model method of the surgical object provided by any of the embodiments of the present disclosure, and has the corresponding function modules and beneficial effects of the execution method.

[0120] It is worth noting that each unit and module included in the above device is only divided according to the function logic, but is not limited to the above division, as long as the corresponding function can be realized; in addition, the specific name of each functional unit is only for the convenience of mutual differentiation, and does not serve to limit the protection scope of the embodiments of the present disclosure.

[0121] Figure 7 A structural schematic diagram of an electronic device 10 that can be used to implement embodiments of the present application is shown. The electronic device is intended to represent various forms of digital computers, such as laptops, desktops, tablets, personal digital assistants, servers, blade servers, mainframes, and other appropriate computers. The electronic device can also represent various forms of mobile devices, such as personal digital processors, cellular telephones, smart phones, wearable devices (e.g., headsets, glasses, watches, etc.), and other similar computing devices. The components shown here, their connections and relationships, and their functions, are meant to be examples only, and are not intended to limit implementations of the present application described and / or claimed in this document.

[0122] As shown in Figure 7 The electronic device 10 includes at least one processor 11, and a memory, such as a read-only memory (ROM) 12, a random access memory (RAM) 13, etc., which is communicatively connected to the at least one processor 11, wherein the memory stores a computer program that can be executed by the at least one processor. The processor 11 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 12 or the computer program loaded from the storage unit 18 into the random access memory (RAM) 13. In the RAM 13, various programs and data required for the operation of the electronic device 10 can also be stored. The processor 11, the ROM 12, and the RAM 13 are connected to each other through a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0123] A plurality of components in the electronic device 10 are connected to the I / O interface 15, including: an input unit 16, such as a keyboard, a mouse, etc.; an output unit 17, such as various types of displays, speakers, etc.; a storage unit 18, such as a magnetic disk, an optical disk, etc.; and a communication unit 19, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 19 allows the electronic device 10 to exchange information / data with other devices through a computer network, such as the Internet, and / or various telecommunications networks.

[0124] The processor 11 can be various general and / or special purpose processing components with processing and computing capabilities. Some examples of the processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various specialized artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The processor 11 performs various methods and processes described above, such as the generation of the postoperative three-dimensional model of the surgical object.

[0125] In some embodiments, the generation of the postoperative three-dimensional model of the surgical object can be implemented as a computer program tangibly embodied in a computer readable storage medium, such as the storage unit 18. In some embodiments, part or all of the computer program can be loaded onto and / or installed on the electronic device 10 via the ROM 12 and / or the communication unit 19. When the computer program is loaded onto the RAM 13 and executed by the processor 11, one or more steps of the method of the generation of the postoperative three-dimensional model of the surgical object described above can be performed. Alternatively, in other embodiments, the processor 11 can be configured to perform the generation of the postoperative three-dimensional model of the surgical object by any other suitable means, such as by means of firmware.

[0126] Various implementations of the systems and techniques described above can be realized in digital electronic circuitry, integrated circuitry, a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), a system on a chip (SOC), a programmable logic device (PLD), a computer hardware, firmware, software, and / or combinations thereof. These various implementations can include implementation in one or more computer programs that are executable and / or interpretable on a programmable system including at least one programmable processor, which can be special or general purpose, coupled to receive data and instructions from, and to transmit data and instructions to, a storage system, at least one input device, and at least one output device.

[0127] Computer programs used to implement the methods of the application can be written in any combination of one or more programming languages. These computer programs can be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the computer program, when executed by the processor of the machine, implements the functions / acts specified in the flowcharts and / or block diagrams. The computer program can be executed entirely on a machine, partially on a machine, partially on a machine as a stand-alone software package, partially on a machine and partially on a remote machine or entirely on a remote machine or server.

[0128] In the context of the present application, a computer-readable storage medium can be a tangible medium that can contain or store a computer program for use by or in connection with an instruction execution system, apparatus, or device. A computer-readable storage medium can include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium can be a machine-readable signal medium. More specific examples of a machine-readable storage medium will include one or more lines of a program of instructions in a transitory signal, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0129] To provide for interaction with a user, the systems and techniques described here can be implemented on an electronic device having a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user and a keyboard and a pointing device (e.g., a mouse or a trackball) by which the user can provide input to the electronic device. Other kinds of devices can be used to provide for interaction with a user as well; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form, including acoustic, speech, or tactile input.

[0130] The systems and techniques described here can be implemented in a computing system that includes a back end component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front end component (e.g., a user computer having a graphical user interface or a Web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such back end, middleware, or front end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network (LAN), a wide area network (WAN), blockchain network, and the Internet.

[0131] The computing system can include a user terminal and a server. The user terminal and the server are generally remote from each other and typically interact through a communication network. The relationship of user terminal and server is created by computer programs running on the respective computers and having a user terminal-server relationship to each other. The server can be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the defects of large management difficulty and weak business scalability in traditional physical host and VPS service.

[0132] It should be understood that the steps shown above can be reordered, added to, or deleted from. For example, the steps described in the present application can be executed in parallel, sequentially, or in a different order, as long as the desired results of the technical solutions of the present application can be achieved, and the present application is not limited herein.

[0133] The embodiment provides a computer readable storage medium, which stores a computer program, and the program is executed by a processor to realize a postoperative three-dimensional model generation step of a surgical object provided by any embodiment of the present application, and the method comprises the following steps:

[0134] A first three-dimensional model of the surgical object is acquired by a three-dimensional scanning system, and a second three-dimensional model of the surgical object is acquired by a medical imaging technology;

[0135] A surgical correction model of the surgical object is determined based on the second three-dimensional model;

[0136] The first three-dimensional model is corrected based on the surgical correction model, so as to obtain the postoperative three-dimensional model of the surgical object.

[0137] The computer storage medium of the embodiment of the present application can adopt any combination of one or more computer readable media. The computer readable medium can be a computer readable signal medium or a computer readable storage medium. The computer readable storage medium may, for example, but is not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or component, or any combination of the above. More specific examples (non-exhaustive list) of the computer readable storage medium include an electrical connection having one or more wires, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this document, the computer readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, device or component.

[0138] A computer readable signal medium can include a propagated data signal with computer executable code embodied therein, for example, in baseband or as part of a carrier wave. Such a propagated signal can take any of a variety of forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof. A computer readable signal medium can be any computer readable medium that can be involved in

[0139] The code can be transmitted in any form, including, but not limited to, radio frequency, electrical, optical, acoustical, or any form of propagation medium. The code may

[0140] The code can be written in any form of programming language, including object oriented programming languages such as Java, Smalltalk, C++, or the like, and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The code can execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the latter scenario, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection can be made to an external computer (for example, through the Internet using an Internet Service Provider). The application is related to the use of computer program code

[0141] Those skilled in the art will appreciate that the modules or steps of the application described above can be implemented in computer languages such as Java, C++, or the like, and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The code can execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the latter scenario, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection can be made to an external computer (for example, through the Internet using an Internet Service Provider). The application is related to the use of computer program code

[0142] It should be understood that the processes shown in the figures, including the steps of the processes, can be re-ordered, added to, or removed from, without departing from the scope of the application. For example, the steps of the processes described in the application can be performed in parallel, in series, or in a different order, without departing from the scope of the application, and are not limited herein.

[0143] The above detailed description does not limit the scope of the application. Various modifications, combinations, sub-combinations and alternatives can be apparent to one skilled in the art without departing from the spirit and the principles of the application. Any modification, equivalent replacement and improvement etc. made within the spirit and principles of the application shall be included in the scope of the protection of the application.

Claims

1. A method of generating a postoperative three-dimensional model of a surgical subject, characterized by, The method comprises the following steps: obtaining a first three-dimensional model of the surgical object through a three-dimensional scanning system, and obtaining a second three-dimensional model of the surgical object through a medical imaging technology; determining a surgical correction model of the surgical object based on the second three-dimensional model; correcting the first three-dimensional model based on the surgical correction model to obtain a postoperative three-dimensional model of the surgical object; wherein the correction of the first three-dimensional model based on the surgical correction model comprises: registering the first three-dimensional model and the second three-dimensional model to determine model registration information between the first three-dimensional model and the second three-dimensional model, the model registration information being a geometric fitting model between the first three-dimensional model and the second three-dimensional model, and the model registration information being used for correcting the first three-dimensional model; correcting the first three-dimensional model based on the surgical correction model and the model registration information, comprising: determining a first reference point in the surgical correction model, and determining first position transformation information of the first reference point in the second three-dimensional model and the surgical correction model; determining second position transformation information corresponding to each surface pixel point in the first three-dimensional model based on the model registration information and the first position transformation information; performing position transformation on each surface pixel point based on the second position transformation information corresponding to the surface pixel point.

2. The method of claim 1, wherein, The registration of the first three-dimensional model and the second three-dimensional model comprises: obtaining a plurality of first marker points in the first three-dimensional model and second marker points corresponding to the first marker points in the second three-dimensional model; determining the model registration information between the first three-dimensional model and the second three-dimensional model based on the first marker points, the second marker points and a preset model registration algorithm, wherein the model registration algorithm comprises at least one of affine transformation, perspective transformation and polynomial transformation.

3. The method of claim 2, wherein, After determining the model registration information between the first three-dimensional model and the second three-dimensional model, the method further comprises: updating the model registration information between the first three-dimensional model and the second three-dimensional model based on an iterative nearest neighbor point algorithm.

4. The method of claim 1, wherein, The determination of the second position transformation information corresponding to each surface pixel point in the first three-dimensional model based on the model registration information and the first position transformation information comprises: determining a second reference point in the first three-dimensional model corresponding to the first reference point based on the model registration information; determining distance information between the surface pixel point and the second reference point for each surface pixel point in the first three-dimensional model; determining the second position transformation information corresponding to the surface pixel point based on the distance information and the first position transformation information.

5. The method of claim 4, wherein, The determination of the second position transformation information corresponding to the surface pixel point based on the distance information and the first position transformation information comprises: determining a weight corresponding to the surface pixel point based on the distance information, wherein the weight and the distance information are in a positive correlation. Determine second position transformation information corresponding to the surface pixel point based on the weight and the first position transformation information.

6. The method of claim 1, wherein, The surgical correction model comprises a bone correction model; The surgical correction model of the surgical object is determined based on the second three-dimensional model, comprising: Segmenting the second three-dimensional model to obtain a bone three-dimensional model of the surgical object; The bone three-dimensional model is corrected to obtain a bone correction model of the surgical object.

7. The method of claim 6, wherein, The bone three-dimensional model is corrected to obtain a bone correction model of the surgical object, comprising: The bone three-dimensional model is corrected to obtain a preliminary correction model of the surgical object; The preliminary correction model is subjected to three-dimensional finite element analysis to determine the correction effect of the preliminary correction model. If the correction effect meets the preset condition, the preliminary correction model is taken as the bone correction model of the surgical object.

8. An apparatus for generating a postoperative three-dimensional model of a surgical subject, characterized by comprising: Comprise: The preoperative model acquisition module is used for acquiring the first three-dimensional model of the surgical object through a three-dimensional scanning system, and acquiring the second three-dimensional model of the surgical object through a medical imaging technology; The surgical correction model determination module is used for determining the surgical correction model of the surgical object based on the second three-dimensional model; The postoperative model generation module is used for correcting the first three-dimensional model based on the surgical correction model to obtain a postoperative three-dimensional model of the surgical object; The postoperative model generation module is specifically used for registering the first three-dimensional model and the second three-dimensional model to determine model registration information between the first three-dimensional model and the second three-dimensional model, the model registration information is a geometric fitting model between the first three-dimensional model and the second three-dimensional model, and the model registration information is used for correcting the first three-dimensional model; The first three-dimensional model is corrected based on the surgical correction model and the model registration information; The postoperative model generation module is specifically used for determining a first reference point in the surgical correction model, determining first position transformation information of the first reference point in the second three-dimensional model and the surgical correction model, determining second position transformation information corresponding to each surface pixel point in the first three-dimensional model based on the model registration information and the first position transformation information, and performing position transformation on each surface pixel point based on the second position transformation information corresponding to each surface pixel point.

Images