Method, device, computer device and storage medium for registering tissue organs

Abstract

The application relates to a registration method and device of an organ, a computer device and a storage medium. The method comprises the following steps: obtaining a first processing image of a first image and a second processing image of a second image, performing rigid registration on a target organ in the first processing image and a target organ in the second processing image to obtain a rigid registration result, performing elastic registration on the target organ in the rigid registration result and the target organ in the second processing image to obtain an elastic registration result, and processing the first image according to the rigid registration result and the elastic registration result to obtain a registered image. The method can register rich information of the target organ contained in the first image to the second image, and the registration is based on the conversion relationship of feature points in the target organ. Compared with a traditional registration method which is based on only surface feature points of the target organ, the registration result is more accurate and has strong robustness.

Description

Technical Field

[0001] This application relates to the field of medical image processing technology, and in particular to a method, apparatus, computer device, and storage medium for registering tissues and organs. Background Technology

[0002] During interventional thoracic and abdominal surgeries, preoperative diagnosis typically involves contrast-enhanced CT scans to provide anatomical information about organs, blood vessels, and tumors. However, due to clinical limitations, intraoperative plain CT scans are usually performed. Environmental factors and patient conditions can cause displacement of organs and tissues in intraoperative plain CT images compared to preoperative contrast-enhanced images. Therefore, the preoperative anatomical information cannot be utilized intraoperatively, significantly increasing the difficulty and challenge of the interventional procedure. Thus, preoperative and intraoperative registration is necessary to transfer the preoperative anatomical information to the intraoperative setting, reducing the complexity and challenge of the interventional surgery.

[0003] Traditional methods for registering preoperative CT enhanced images to intraoperative CT plain images include: extracting features from the preoperative CT enhanced image and the intraoperative CT plain image, and then registering the features from the preoperative CT enhanced image and the CT plain image to complete the registration of features from the preoperative CT enhanced image to the intraoperative CT plain image.

[0004] However, the above registration method requires symmetrical data information between the two sets of data to be registered. However, the preoperative enhanced CT images of interventional surgery have a lot of contrast-enhanced information, while the intraoperative plain CT images lack this information. As a result, the registration results are poor for detailed information inside organs, such as blood vessels and tumors, during the registration process, which cannot meet the high accuracy requirements of actual interventional surgery. Summary of the Invention

[0005] Therefore, it is necessary to provide a registration method, apparatus, computer equipment, and storage medium for tissues and organs that can improve registration accuracy in response to the above-mentioned technical problems.

[0006] Firstly, a method for registering tissues and organs, the method comprising:

[0007] Acquire a first processed image of the first image and a second processed image of the second image; the first image and the second image contain the target organ of the same object at different times;

[0008] The target organ in the first processed image and the target organ in the second processed image are rigidly registered to obtain a rigid registration result;

[0009] The target organ in the rigid registration result and the target organ in the second processed image are elastically registered to obtain the elastic registration result.

[0010] The first image is processed based on the rigid registration result and the elastic registration result to obtain the registered image.

[0011] In one embodiment, processing the first image based on the rigid registration result and the elastic registration result to obtain the registered image includes:

[0012] A mechanical model of the target organ is constructed based on the rigid registration results.

[0013] The difference between the elastic registration result and the rigid registration result is used as the boundary condition to solve the mechanical model, thereby obtaining the deformation field of each point inside the target organ;

[0014] The first image is processed based on the rigid transformation matrix in the rigid registration result and the deformation field of each point inside the target organ to obtain the registered image.

[0015] In one embodiment, processing the first image based on the rigid transformation matrix in the rigid registration result and the deformation field of each point inside the target organ to obtain the registered image includes:

[0016] Based on the rigid transformation matrix, the first image is translated and / or rotated to obtain the transformed first image;

[0017] Based on the deformation field of each point inside the target organ, each point inside the target organ in the transformed first image is translated and / or rotated to obtain the registered image.

[0018] In one embodiment, the method further includes:

[0019] The deformation field of each point inside the target organ is interpolated to obtain the deformation field of each pixel inside the target organ.

[0020] The process of translating points within the target organ in the transformed first image based on the deformation field of each point inside the target organ to obtain the registered image includes:

[0021] Based on the deformation field of each pixel inside the target organ, each pixel inside the target organ in the transformed first image is translated to obtain the registered image.

[0022] In one embodiment, acquiring the first processed image of the first image and the second processed image of the second image includes:

[0023] Obtain the first image and the second image;

[0024] Extract the first mask image of the target organ from the first image, and the second mask image of the target organ from the second image;

[0025] The target organ in the first mask image is divided into a surface mesh to obtain the first processed image, and the target organ in the second mask image is divided into a surface mesh to obtain the second processed image.

[0026] In one embodiment, the first image is an enhanced image, and the second image is a plain scan image. Before acquiring the first processed image of the first image and the second processed image of the second image, the method further includes:

[0027] Acquire a third image; the third image is an enhanced image.

[0028] The third image is transformed to obtain the second image.

[0029] In one embodiment, the step of performing image transformation on the third image to obtain the second image includes:

[0030] The third image is processed to remove highlight information, thus obtaining the second image;

[0031] Alternatively, the third image can be input into a preset image transformation network to obtain the second image.

[0032] Secondly, a method for registering and verifying tissues and organs, the method comprising:

[0033] Extract the target organ and its internal details mask from the third image to obtain a third mask image corresponding to the third image; the third image is an enhanced image.

[0034] The registered image is verified based on the third mask image; the registered image is the image obtained by registering the first image and the second image according to the method described in the first aspect; the second image is the image obtained by transforming the third image.

[0035] In one embodiment, verifying the registered image based on the third mask image includes:

[0036] Extract the target organ and internal detail mask from the registered image to obtain a fourth mask image corresponding to the registered image;

[0037] Calculate the number of pixels that overlap between the target organ region in the third mask image and the target organ region in the fourth mask image;

[0038] The verification value is obtained based on the number of repeated pixels, the number of pixels in the target organ region in the third mask image, and the number of pixels in the target organ region in the fourth mask image.

[0039] If the verification value is greater than a preset threshold, the registered image is determined to have passed verification; if the verification value is not greater than the preset threshold, the registered image is determined to have failed verification.

[0040] Thirdly, a registration device for tissues and organs, the device comprising:

[0041] Acquire a first processed image of a first image and a second processed image of a second image; the first image and the second image contain the target organ of the same object at different times;

[0042] The target organ in the first processed image and the target organ in the second processed image are rigidly registered to obtain a rigid registration result;

[0043] The target organ in the rigid registration result and the target organ in the second processed image are elastically registered to obtain the elastic registration result.

[0044] The first image is processed based on the rigid registration result and the elastic registration result to obtain the registered image.

[0045] Fourthly, a registration and verification device for tissues and organs, the device comprising:

[0046] The segmentation module is used to extract target organs and internal detail masks from the third image to obtain a third mask image corresponding to the third image; the third image is an enhanced image.

[0047] The verification module is used to verify the registered image based on the third mask image; the registered image is the image obtained by registering the first image and the second image according to the apparatus described in the third aspect.

[0048] Fifthly, a computer device includes a memory and a processor, the memory storing a computer program, the processor executing the computer program to implement the methods described in the first and second aspects above.

[0049] A sixth aspect is a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the methods described in the first and second aspects above.

[0050] The above-mentioned organ registration method, apparatus, computer equipment, and storage medium acquire a first processed image of a first image and a second processed image of a second image, then perform rigid registration on the target organ in the first processed image and the target organ in the second processed image to obtain a rigid registration result, and perform elastic registration on the target organ in the rigid registration result and the target organ in the second processed image to obtain an elastic registration result, and then process the first image according to the rigid registration result and the elastic registration result to obtain a registered image. The above method provides a registration verification method for different types of images. It obtains the approximate registration position of the target organ through rigid registration and the registration position of the surface feature points of the target organ through elastic registration. Then, based on the approximate registration position and the registration position of the surface feature points of the target organ, the transformation relationship of each feature point inside the target organ in the first image and the second image to be registered can be calculated. Based on the transformation relationship, the first image to be registered is then registered, which can realize the registration of the rich information of the target organ contained in the first image into the second image. Moreover, the registration based on the transformation relationship of each feature point inside the target organ can avoid the problem of inaccurate results caused by traditional pixel-based registration when the pixel information of the enhanced image and the flat scan image is asymmetrical. Therefore, the registration result obtained by the registration method provided in this application is more accurate and robust. Attached Figure Description

[0051] Figure 1 This is a diagram illustrating the application environment of a tissue and organ registration method in one embodiment.

[0052] Figure 2 This is a flowchart illustrating a registration method for tissues and organs in one embodiment;

[0053] Figure 3 Figure 2 A flowchart illustrating one implementation of S104 in the embodiment;

[0054] Figure 4 Figure 3 A flowchart illustrating one implementation of S203 in the embodiment;

[0055] Figure 5 Figure 3 A flowchart illustrating one implementation of S203 in the embodiment;

[0056] Figure 6 This is a flowchart illustrating a registration method for tissues and organs in one embodiment;

[0057] Figure 7 Figure 6 A flowchart illustrating one implementation of S401 in the embodiment;

[0058] Figure 8 This is a flowchart illustrating a registration and verification method for tissues and organs in one embodiment.

[0059] Figure 9 Figure 8 A flowchart illustrating one implementation of S602 in the embodiment;

[0060] Figure 10 This is a flowchart illustrating a registration and verification method for tissues and organs in one embodiment.

[0061] Figure 11 This is a structural block diagram of a registration device for tissues and organs in one embodiment;

[0062] Figure 12 This is a structural block diagram of a registration device for tissues and organs in one embodiment;

[0063] Figure 13 This is a structural block diagram of a registration device for tissues and organs in one embodiment;

[0064] Figure 14 This is a structural block diagram of a registration device for tissues and organs in one embodiment;

[0065] Figure 15 This is a structural block diagram of a registration device for tissues and organs in one embodiment;

[0066] Figure 16 This is a structural block diagram of a registration device for tissues and organs in one embodiment;

[0067] Figure 17 This is a structural block diagram of a registration and verification device for tissues and organs in one embodiment;

[0068] Figure 18 This is a structural block diagram of a registration and verification device for tissues and organs in one embodiment;

[0069] Figure 19 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0070] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0071] In practical applications, preoperative diagnosis for thoracic and abdominal interventional surgeries typically involves contrast-enhanced CT scans to obtain enhanced images. These images provide clear anatomical information about organs, blood vessels, and tumors, enabling accurate segmentation of these structures. However, due to clinical limitations, intraoperative plain CT scans are generally performed only. Therefore, intraoperative plain CT images lack the anatomical information about blood vessels and tumors required for interventional surgery. Blood vessels are crucial tissues to avoid during surgical path planning, and tumors are the targets; both are essential tissues and lesions to identify for intraoperative surgical path planning and treatment guidance. Furthermore, environmental factors, the patient's physical condition, and respiratory movements cause shifts in the intraoperative plain CT images compared to the preoperative contrast-enhanced CT images, making it impossible to utilize the rich preoperative information intraoperatively, significantly increasing the difficulty and challenge of interventional surgery. Therefore, preoperative and intraoperative registration is necessary to transfer the abundant preoperative information to the intraoperative process, reducing the difficulty and challenge of interventional surgery. Traditional voxel- or feature-based registration methods require symmetrical data from both sets of data. However, preoperative images for interventional surgery contain a large amount of contrast-enhanced highlighting information, while intraoperative plain scans lack this information. This results in poor registration of internal organ details, such as blood vessels and tumors, during the registration process, failing to meet the high accuracy requirements of actual interventional surgeries. To address these technical problems, this application proposes a tissue and organ registration verification method. Based on biomechanical models and surface elastic registration methods, it realistically simulates the deformation of the organ surface and interior during registration. By utilizing simulated real deformation, it ensures accurate registration of internal organ details, such as blood vessels and tumors, meeting the high accuracy requirements of actual interventional surgeries. The following embodiments will detail the tissue and organ registration verification method provided in this application.

[0072] The tissue and organ registration verification method provided in this application can be applied to, for example... Figure 1 In the illustrated application environment, terminal 102 and server 104 are connected via wireless or wired communication. Terminal 102 is used to acquire two image data sets to be registered, i.e., image data containing the target organ at different times. Server 104 is used to acquire the two image data sets to be registered from terminal 102 and process them to complete the registration of the target organ in the two image data sets. Terminal 102 can be any type of image scanning device, such as a computed tomography (CT) scanner or a magnetic resonance (MR) scanner. Server 104 can be, but is not limited to, various personal computers, laptops, smartphones, tablets, and portable wearable devices. Server 104 can also be implemented using a standalone server or a server cluster consisting of multiple servers.

[0073] Those skilled in the art will understand that Figure 1 The application environment shown in the figure is merely a block diagram of a partial structure related to the solution of this application, and does not constitute a limitation on the application environment to which the solution of this application is applied. The specific application environment may include more or fewer components than shown in the figure, or a combination of certain components, or different component arrangements.

[0074] In one embodiment, such as Figure 2 As shown, a registration and verification method for tissues and organs is provided, which can be applied to... Figure 1 Taking the server in the example, the following steps are included:

[0075] S101, acquire the first processed image of the first image and the second processed image of the second image.

[0076] The first and second images contain the target organ or tissue of the same object at different times. For example, the first and second images are different types of image data of the same organ collected from the same patient at different times. The first image can be enhanced image data, and the second image can be plain scan image data. The target organ can be any type of tissue or organ, such as the lung, liver, or spleen, and is not limited here. Specifically, the first and second images are regional images or slice images of the corresponding target organ / tissue at different times, as well as images of any modality (e.g., CT, MR, ultrasound, etc.) scanned at different times. The first processed image of the first image includes a mesh image after the tissue and organ in the first image are meshed, and may also include a mask image after mask extraction of the first image.

[0077] In this embodiment, a CT scanner can be used to pre-scan and perform enhanced and plain scans of the same object to obtain enhanced and plain image data corresponding to the same object. The server obtains the enhanced and plain image data of the same object from the CT scanner, i.e., the first image and the second image. For example, the server can obtain enhanced image data of patient A's chest and abdomen before surgery to obtain the first image, and obtain plain image data of patient A's chest and abdomen during surgery to obtain the second image. Optionally, the server can also obtain image data of the target organ of the same object at different times through other means, such as directly downloading the first and second images corresponding to the same object from the Internet. After the server obtains the first and second images, it can also process the first and second images in the same way, such as surface reconstruction and mesh generation, to obtain their respective mesh images, i.e., the first processed image corresponding to the first image and the second processed image corresponding to the second image; optionally, after the server obtains the first and second images, it can also perform mask extraction on the first and second images respectively to obtain their respective mask images, i.e., the first processed image corresponding to the first image and the second processed image corresponding to the second image.

[0078] S102, rigidly register the target organ in the first processed image and the target organ in the second processed image to obtain the rigid registration result.

[0079] The rigid registration result includes a rigid transformation matrix and a rigidly registered image. The rigid transformation matrix represents the distance each point on the target organ moves when the target organ's position in the first processed image is moved to its position in the second processed image during rigid registration. The rigidly registered image is the image of the target organ after horizontal or vertical movement or deflection of the target organ in the first processed image according to its position in the second processed image, or the image of the target organ after horizontal or vertical movement or deflection of the target organ in the second processed image according to its position in the first processed image.

[0080] In this embodiment, when the server acquires the first processed image and the second processed image, it can perform rigid registration on the target organ in the first processed image and the target organ in the second processed image to obtain an image including a rigid transformation matrix and rigid registration. Specifically, during the rigid registration process, the server can use the location of the target organ in the second processed image as a reference and perform translation, deflection, or other operations on the target organ in the first processed image in different directions to align the location of the target organ in the first processed image with the location of the target organ in the second processed image. Optionally, the server can also use the location of the target organ in the first processed image as a reference and perform translation, deflection, or other operations on the target organ in the second processed image in different directions to align the location of the target organ in the second processed image with the location of the target organ in the first processed image.

[0081] S103, perform elastic registration on the target organ in the rigid registration result and the target organ in the second processed image to obtain the elastic registration result.

[0082] When the server obtains the rigid registration result, it can obtain the rigidly registered image. Furthermore, it can perform elastic registration on the target organ in the rigidly registered image and the target organ in the second processed image to obtain an elastic registration result including the elastically registered image. Specifically, during elastic registration, the server can perform translation operations on the feature points on the surface of the target organ in the rigidly registered image and the target organ in the second processed image in either the front-back or depth direction. This ensures that the positions of the feature points on each surface of the target organ in the rigidly registered image coincide with the positions of the feature points on each surface of the target organ in the second processed image.

[0083] S104. The first image is processed according to the rigid registration result and the elastic registration result to obtain the registered image.

[0084] Once the server obtains the rigid registration result and the elastic registration result, since the rigid registration result contains the transformation relationship of the target organ's location during the change from the target organ in the first image to the target organ in the second image, and since the elastic registration result contains the transformation relationship of each surface feature point on the target organ from the target organ in the first image to the target organ in the second image, by analyzing the relationship of the sequential change of the target organ in the rigid registration result and the elastic registration result, the transformation relationship of each feature point inside the target organ from the target organ in the first image to the target organ in the second image can be obtained. Based on this transformation relationship, each feature point on the target organ in the first image is transformed, and the transformed image is determined as the registered image, thus realizing the registration of the rich information in the first image to the second image.

[0085] The above-mentioned organ registration method involves acquiring a first processed image of a first image and a second processed image of a second image, then performing rigid registration on the target organ in the first processed image and the target organ in the second processed image to obtain a rigid registration result, and then performing elastic registration on the target organ in the rigid registration result and the target organ in the second processed image to obtain an elastic registration result. Finally, the first image is processed based on the rigid registration result and the elastic registration result to obtain a registered image. The above method provides a registration verification method for different types of images. It obtains the approximate registration position of the target organ through rigid registration and the registration position of the surface feature points of the target organ through elastic registration. Then, based on the approximate registration position and the registration position of the surface feature points of the target organ, the transformation relationship of each feature point inside the target organ in the first image and the second image to be registered can be calculated. Based on the transformation relationship, the first image to be registered is then registered, which can realize the registration of the rich information of the target organ contained in the first image into the second image. Moreover, the registration based on the transformation relationship of each feature point inside the target organ can avoid the problem of inaccurate results caused by traditional pixel-based registration when the pixel information of the enhanced image and the flat scan image is asymmetrical. Therefore, the registration result obtained by the registration method provided in this application is more accurate and robust.

[0086] In one embodiment, one implementation of S104 above is provided, such as... Figure 3 As shown, the above-mentioned S104 "processing the first image according to the rigid registration result and the elastic registration result to obtain the registered image" includes:

[0087] S201, construct a mechanical model of the target organ based on the target organ in the rigid registration results.

[0088] Once the server obtains the rigid registration result, it can obtain the approximate outline and volume data of the target organ in the first image. The computer device can then construct the mechanical model of the target organ in the simulation software based on this data and the corresponding mechanical model construction algorithm.

[0089] S202 uses the difference between the elastic registration result and the rigid registration result as the boundary condition to solve the mechanical model and obtain the deformation field of each point inside the target organ.

[0090] When the server obtains the rigid registration result and the elastic registration result, since the rigid registration result is the approximate position matching of each point on the target organ and the elastic registration result is the approximate position matching of each point on the surface of the target organ, the difference between the elastic registration result and the rigid registration result can reflect the positional change relationship of each point on the surface of the target organ. Based on this, in this embodiment, the difference between the elastic registration result and the rigid registration result can be used as the boundary condition, and combined with the corresponding material parameters, the deformation field of each point inside the target organ can be obtained by solving the mechanical model of the target organ.

[0091] S203, the first image is processed based on the rigid transformation matrix in the rigid registration result and the deformation field of each point inside the target organ to obtain the registered image.

[0092] When the server obtains the rigid transformation matrix and the deformation field of each point inside the target organ in the rigid registration result, it can first perform coarse registration on each point of the target organ in the first image according to the rigid transformation matrix to obtain a preliminary registered image. Then, it can perform detailed registration on the internal points of the target organ in the preliminary registered image according to the deformation field of each point inside the target organ to finally obtain the registered image.

[0093] The above method achieves accurate registration of points on the target organ in different types of images by solving the mechanical model of the target organ using the difference between the elastic registration result and the rigid registration result as the boundary condition. It provides a method for registration based on a biomechanical model. By solving the mechanical model of the target organ using the finite element method, the deformation field of each point inside the target organ can be obtained. The registration method based on this deformation field can make the registered image contain rich feature information from the first image. Therefore, the registration method provided in this application can be applied to the accurate registration of preoperative enhanced images and intraoperative plain scan images, so that the intraoperative plain scan image can contain more highlight information from the preoperative enhanced image. This overcomes the problem of high surgical failure rate caused by the lack of highlight information in the plain scan image when doctors refer to it for surgery. Therefore, when the registration verification method provided in this application is applied to preoperative and intraoperative image registration, it can improve the success rate of surgery.

[0094] Furthermore, such as Figure 4 As shown, the above-mentioned step S203, "processing the first image according to the rigid transformation matrix in the rigid registration result and the deformation field of each point inside the target organ to obtain the registered image," includes:

[0095] S301, perform translation and / or rotation transformations on the first image based on the rigid transformation matrix to obtain the transformed first image.

[0096] Specifically, when the server performs coarse registration of the target organ in the first image based on the rigid transformation matrix, it can perform translation and / or rotation transformations on each point on the target organ in the first image to obtain the transformed first image.

[0097] S302, based on the deformation field of each point inside the target organ, translate and / or rotate each point inside the target organ in the transformed first image to obtain the registered image.

[0098] Specifically, when the server performs fine registration of each point inside the target organ in the first image based on the deformation field of each point inside the target organ, it can perform translation and / or rotation transformations on each point inside the target organ in the first image to obtain the registered image.

[0099] In one embodiment, such as Figure 5 As shown above, Figure 4 The method described in the embodiments further includes the following steps:

[0100] S303, interpolate the deformation field of each point inside the target organ to obtain the deformation field of each pixel inside the target organ.

[0101] When the server performs fine registration of each point inside the target organ in the first image based on the deformation field of each point inside the target organ, in order to register the first image more accurately, the server can first perform interpolation processing on the deformation field of each point inside the target organ to obtain the deformation field of each pixel inside the target organ.

[0102] Correspondingly, when the server executes the above S302 step, it specifically performs the following: based on the deformation field of each pixel point inside the target organ, it translates each pixel point inside the target organ in the transformed first image to obtain the registered image.

[0103] Specifically, when the server performs fine registration of each point inside the target organ in the first image based on the deformation field of each pixel inside the target organ, it can translate and / or rotate each pixel inside the target organ in the first image to obtain the registered image. Since the pixels inside the target organ are more densely packed than the pixels inside the established mechanical model of the target organ, interpolating the deformation field of each point inside the mechanical model of the target organ to make it consistent with the pixel density inside the target organ allows the interpolated deformation field of each point to more accurately reflect the changes of each feature point inside the target organ. Therefore, the registration method provided in this embodiment can obtain more accurate registration results and make the registered image contain richer highlight information.

[0104] Optional, such as Figure 6 As shown, before the server obtains the first processed image of the first image and the second processed image of the second image, the following steps need to be performed:

[0105] S401, acquire the first image and the second image.

[0106] The first image can be enhanced image data, and the second image can be plain scan image data. Optionally, the first image can also be other types of image data, and the second image can also be other types of image data, as long as the first image and the second image contain the same scanned object.

[0107] In this embodiment, a CT scanner can be used to pre-scan the same object with contrast and without contrast, obtaining contrast and non-contrast image data corresponding to the same object. The server then obtains this contrast and non-contrast image data, i.e., the first image and the second image, from the CT scanner by connecting to it. For example, the server can obtain contrast image data of patient A's chest and abdomen before surgery to obtain the first image, and obtain non-contrast image data of patient A's chest and abdomen during surgery to obtain the second image. Optionally, the server can also obtain image data of the target organs of the same object at different times through other methods.

[0108] S402, extract the first mask image of the target organ from the first image, and extract the second mask image of the target organ from the second image.

[0109] In this embodiment, when the server obtains the first image and the second image, it can further use a preset surface reconstruction method to extract the mask image of the target organ in the first image to obtain the first mask image, and extract the mask image of the target organ in the second image to obtain the second mask image.

[0110] S403, perform surface mesh division on the target organ in the first mask image to obtain a first processed image, and perform surface mesh division on the target organ in the second mask image to obtain a second processed image.

[0111] In order to establish a mechanical model of the target organ in the first image and to perform rigid and elastic registration of the first image, when the server obtains the first mask image corresponding to the first image and the second mask image corresponding to the second image, it is necessary to further divide the first mask image into a mesh, divide the surface of the target organ in the first mask image into a mesh, and obtain the data of each feature point on the upper surface of the target organ. Similarly, the second mask image is divided into a mesh, and the surface of the target organ in the second mask image is divided into a mesh, and the data of each feature point on the upper surface of the target organ is obtained. This is so that a mechanical model of the target organ can be constructed based on the data of each feature point on the upper surface of the target organ, and rigid and elastic registration of each surface point of the target organ in the first mask image and the target organ in the second mask image can be achieved.

[0112] Optionally, one implementation method for obtaining the second image mentioned above is provided, such as... Figure 7 As shown, this method includes:

[0113] S501, Obtain the third image; the third image is an enhanced image.

[0114] Based on the aforementioned S101, the server can obtain a first image and a second image of the same object at different times. The first image is an enhanced image, and the second image is a plain scan image. Correspondingly, the server can also obtain a first image and a third image of the same object at different times. The third image is also an enhanced image, but it is at a different time than the first image. For example, the first image is a CT enhanced scan image of the patient before surgery, and the third image is a CT enhanced scan image of the patient during surgery.

[0115] S502, perform image transformation on the third image to obtain the second image.

[0116] When the server can obtain the enhanced third image, it needs to further convert the third image into a flat scan image, i.e., the second image.

[0117] In this embodiment, three methods are provided for obtaining a second image from a third image and obtaining a flat-scan image from an enhanced image. Specifically, the first method is to remove highlight information from the third image to obtain the second image; the second method is to input the third image into a preset image transformation network to obtain the second image; and the third image is to perform image fusion of different phases to obtain the second image.

[0118] In the first approach, the server can segment the internal details of the target organ, such as blood vessels and tumors, from the enhanced image (third image) to determine the highlight positions of the segmented mask image Mask in the original enhanced image of the target organ's internal details. The server then uniformly assigns the pixel values ​​of the non-contrast highlight areas within a certain range around the highlight positions to the highlight positions, removes the highlight information in the enhanced image, and generates a flat scan image, i.e., the second image, based on the enhanced image.

[0119] In the second approach, the server can use a preset algorithm to remove the highlight information of angiography in the third image and simulate the generation of a plain scan image. The preset algorithm can be trained by learning a large number of plain scan images through a generative adversarial network. When the third image is imported, the adversarial network will output a plain scan image corresponding to the third image, i.e., the second image.

[0120] In the third method, the computer device can acquire images of different phases in advance, and then fuse the images of different phases to obtain the fused image, i.e., the second image.

[0121] The method described in the above embodiments proposes a method for generating a plain scan image based on an enhanced image. Because it is often difficult to obtain intraoperative plain scan images of the same scanned object during experiments, when proposing a registration method, registration can be performed first based on the plain scan image converted from the enhanced image. Then, the registered image can be verified based on the enhanced image before conversion to verify whether the registration method provided in this application is feasible and accurate.

[0122] In practical applications, when the server adopts Figures 1-7 After registering the first image and the second image using the registration method described in the embodiment, further registration can be performed on the second image. Figures 8-9 The registration method described in the embodiments is verified to check its feasibility and accuracy. Based on this, this application also provides a registration verification method for tissues and organs, such as... Figure 8 As shown, the method includes:

[0123] S601, extract the target organ and internal detail mask from the third image to obtain the third mask image corresponding to the third image.

[0124] The third image is an enhanced image. Internal details can refer to the internal details of the object being detected in the third image; for example, if the object is lung tissue, internal details could include pulmonary arteries, lesions, etc. This embodiment involves server verification. Figures 1-7 The registration method described in the embodiment involves the following steps: First, the server acquires a third image and uses a corresponding masking algorithm to extract the target organ and internal detail mask from the third image, obtaining a third mask image corresponding to the third image. This third mask image is then used to verify the registration method.

[0125] S602, Verify the registered image based on the third mask image.

[0126] The registered images are the first image and the second image, according to... Figures 1-7 The image obtained after registration using the registration method described in the embodiment. The second image is the image obtained after image transformation of the third image; the specific transformation method can be found in the foregoing. Figure 7 The methods described in the embodiments are not repeated here.

[0127] In this embodiment, when the server obtains the registered image, it can further verify the registered image using the third mask image corresponding to the third image. Since the registered image is obtained by registering the first image and the second image, and the second image is obtained by transforming the third image, masking the third image to obtain the corresponding third mask image, and then using this third mask image to verify the registered image, can effectively and accurately verify the image. Figures 1-7 Whether the registration method described in the embodiments is feasible or accurate.

[0128] Furthermore, in step S602 above, as follows: Figure 9 As shown, it includes:

[0129] S701, extract the target organ and internal detail mask from the registered image to obtain the fourth mask image corresponding to the registered image.

[0130] In this embodiment, the server can use a corresponding mask image processing algorithm to extract the target organ and internal detail mask in the registered image to obtain the fourth mask image.

[0131] S702, calculate the number of pixels that overlap between the target organ region in the third mask image and the target organ region in the fourth mask image.

[0132] When the server obtains the third and fourth mask images, it can begin to verify the fourth mask image. Specifically, the server can compare the positions of each pixel in the target organ region in the third mask image with the positions of each pixel in the target organ region in the fourth mask image, and identify pixels with the same position as duplicate pixels, thereby determining the number of duplicate pixels in the target organ region in the third mask image and the target organ region in the fourth mask image.

[0133] S703, calculate the verification value based on the number of repeated pixels, the number of pixels in the target organ region in the third mask image, and the number of pixels in the target organ region in the fourth mask image. If the verification value is greater than the preset threshold, proceed to step S704. If the verification value is not greater than the preset threshold, proceed to step S705.

[0134] S704, confirming that the registered image verification has passed.

[0135] S705, the image verification after registration failed.

[0136] Furthermore, the server can calculate the number of pixels that overlap between the target organ region in the third mask image and the target organ region in the fourth mask image, the number of pixels in the target organ region in the third mask image, and the number of pixels in the target organ region in the fourth mask image. Optionally, the server can use the following formula (1) to obtain the verification value:

[0137]

[0138] In the above formula, N c N represents the number of pixels that overlap between the target organ region in the third mask image and the target organ region in the fourth mask image; a N represents the number of pixels within the region containing the target organ in the third mask image. b Y represents the number of pixels in the region where the target organ is located in the fourth mask image; Y represents the verification value.

[0139] The aforementioned preset threshold is used to measure the matching degree between the third and fourth mask images, and is determined by the server in advance based on actual matching requirements. When the server obtains the verification value, it can use this verification value to verify the registered image. Specifically, the verification value is compared with the preset threshold. If the verification value is greater than the preset threshold, it indicates that the number of overlapping pixels in the target organ region in the third mask image and the target organ region in the fourth mask image is relatively large, meaning that the matching degree between the third mask image and the registered image is very high, which also indicates that the aforementioned... Figures 1-7 The registration method described in the embodiment is feasible or the registration result is accurate, so in this case, the registered image verification is passed. If the verification value is not greater than the preset threshold, it indicates that the number of overlapping pixels between the target organ region in the third mask image and the target organ region in the fourth mask image is small, that is, the matching degree between the third mask image and the registered image is low, which also indicates the above... Figures 1-7 If the registration method described in the embodiments is not feasible, or if an error occurs in a certain step of the registration process, resulting in inaccurate registered images, then the verification of the registered images is determined to have failed, and the process is returned to the above steps. Figures 1-7 In the registration method described in the embodiment, each step is checked for potential errors, adjusted or corrected, and then the first and second images are re-registered based on the adjusted or corrected steps. Then, the data is used... Figures 8-9 The method described in the embodiments verifies the registration method until the verification is successful.

[0140] Figures 8-9 The tissue and organ registration verification method provided in this embodiment can verify the registration results after registration is completed, thereby verifying the feasibility of the registration method or the accuracy of the registration results, and providing a basis for subsequent... Figures 1-7When the registration method described in the embodiments is applied to clinical medicine, for example, in the registration process of preoperative enhanced images and intraoperative plain scan images, it provides a reliable and accurate registration method to improve the success rate of surgery.

[0141] In summary, all the above embodiments, such as Figure 10 As shown, this application also provides a method for registration and verification of tissues and organs, the method comprising:

[0142] S801, Obtain the third image; the third image is an enhanced image.

[0143] S802, perform image transformation on the third image to obtain the second image.

[0144] S803, acquire the first image; the first image is an enhanced image, and the first image and the third image are images of the same object at different times.

[0145] S804, Extract the first mask image of the target organ from the first image.

[0146] S805, Extract the second mask image of the target organ from the second image.

[0147] S806, Perform surface mesh division on the target organ in the first mask image to obtain the first mesh image.

[0148] S807, perform surface meshing on the target organ in the second mask image to obtain the second mesh image.

[0149] S808, rigidly register the target organ in the first grid image and the target organ in the second grid image to obtain the rigid registration result.

[0150] S809, perform elastic registration on the target organ in the rigid registration result and the target organ in the second mesh image to obtain the elastic registration result.

[0151] S810, constructs a mechanical model of the target organ based on the target organ in the rigid registration results.

[0152] S811 uses the difference between the elastic registration result and the rigid registration result as the boundary condition to solve the mechanical model and obtain the deformation field of each point inside the target organ.

[0153] S812 interpolates the deformation field of each point inside the target organ to obtain the deformation field of each pixel inside the target organ.

[0154] S813, perform translation and / or rotation transformations on the first image based on the rigid transformation matrix to obtain the transformed first image.

[0155] S814, based on the deformation field of each pixel inside the target organ, translates each pixel inside the target organ in the transformed first image to obtain the registered image.

[0156] S815 extracts the target organ and internal detail mask from the registered image to obtain the fourth mask image corresponding to the registered image.

[0157] S816: Extract the target organ and internal detail mask from the third image to obtain the third mask image corresponding to the third image.

[0158] S817, calculate the number of pixels that overlap between the target organ region in the third mask image and the target organ region in the fourth mask image.

[0159] S818: Calculate the verification value based on the number of repeated pixels, the number of pixels in the target organ region of the third mask image, and the number of pixels in the target organ region of the fourth mask image. If the verification value is greater than a preset threshold, the registered image is determined to have passed verification. If the verification value is not greater than the preset threshold, the registered image is determined to have failed verification. Return to step S801 and re-register the first and second images until at least the registered image passes verification.

[0160] Each of the above steps has been described in the foregoing embodiments; please refer to the foregoing descriptions for details, which will not be repeated here. In summary, this application provides a registration and verification method for tissues and organs, that is, a method for registering and verifying different types of images. By verifying the registration method or registration result, and then readjusting the registration method based on the verification result, the registration method provided by this application is made more reliable in practical applications, thereby providing system robustness.

[0161] It should be understood that, although Figure 2-10 The steps in the flowchart are shown sequentially as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order in which these steps are executed, and they can be performed in other orders. Figure 2-10 At least some of the steps in the process may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but may be executed at different times. The execution order of these steps or stages is not necessarily sequential, but may be executed in turn or alternately with other steps or at least some of the steps or stages in other steps.

[0162] In one embodiment, such as Figure 11 As shown, a registration device for tissues and organs is provided, comprising:

[0163] The acquisition module 11 is used to acquire a first processed image of the first image and a second processed image of the second image; the first image and the second image contain the target organ of the same object at different times.

[0164] The rigid registration module 12 is used to rigidly register the target organ in the first processed image and the target organ in the second processed image to obtain a rigid registration result.

[0165] The flexible registration module 13 is used to perform flexible registration between the target organ in the rigid registration result and the target organ in the second processed image to obtain the flexible registration result.

[0166] In one embodiment, such as Figure 12 As shown, the above-mentioned flexible registration module 13 includes:

[0167] Construction unit 131 is used to construct a mechanical model of the target organ based on the target organ in the rigid registration result;

[0168] Solving unit 132 is used to solve the mechanical model by using the difference between the elastic registration result and the rigid registration result as boundary conditions to obtain the deformation field of each point inside the target organ;

[0169] The processing unit 133 is used to process the first image according to the rigid transformation matrix in the rigid registration result and the deformation field of each point inside the target organ to obtain the registered image.

[0170] In one embodiment, such as Figure 13 As shown, the processing unit 133 includes:

[0171] The first transformation subunit 1331 is used to perform translation and / or rotation transformation on the first image based on the rigid transformation matrix to obtain the transformed first image;

[0172] The second transformation subunit 1332 is used to translate and / or rotate each point inside the target organ in the transformed first image based on the deformation field of each point inside the target organ to obtain the registered image.

[0173] In one embodiment, such as Figure 14 As shown, the processing unit 133 further includes:

[0174] Interpolation processing unit 1333 is used to interpolate the deformation field of each point inside the target organ to obtain the deformation field of each pixel point inside the target organ.

[0175] Correspondingly, the second transformation subunit 1332 is specifically used to translate each pixel inside the target organ in the transformed first image based on the deformation field of each pixel inside the target organ to obtain the registered image.

[0176] In one embodiment, such as Figure 15 As shown, the acquisition module 11 includes:

[0177] Acquisition unit 111 is used to acquire the first image and the second image;

[0178] Extraction unit 112 is used to extract a first mask image of the target organ in the first image and a second mask image of the target organ in the second image;

[0179] The partitioning unit 113 is used to partition the target organ in the first mask image into a surface mesh to obtain the first processed image, and to partition the target organ in the second mask image into a surface mesh to obtain the second processed image.

[0180] In one embodiment, such as Figure 16 As shown, the registration device for the aforementioned tissues and organs further includes:

[0181] The initial image acquisition module 14 is used to acquire a third image; the third image is an enhanced image.

[0182] The transformation module 15 is used to perform image transformation on the third image to obtain the second image.

[0183] In one embodiment, the above-mentioned transformation module is specifically used to remove highlight information from the third image to obtain the second image; or, the third image is input into a preset image transformation network to obtain the second image.

[0184] In one embodiment, such as Figure 17 As shown, a registration and verification device for tissues and organs is provided, comprising:

[0185] Segmentation module 21 is used to extract target organs and internal detail masks from the third image to obtain a third mask image corresponding to the third image; the third image is an enhanced image.

[0186] Verification module 22 is used to verify the registered image based on the third mask image; the registered image is the first image and the second image according to... Figures 11-17 The image obtained after registration by the registration device.

[0187] In one embodiment, such as Figure 18 As shown, the verification module 22 includes:

[0188] The segmentation unit 221 is used to extract the target organ and internal detail mask from the registered image to obtain a fourth mask image corresponding to the registered image.

[0189] The calculation unit 222 is used to calculate the number of pixels that overlap between the target organ region in the third mask image and the target organ region in the fourth mask image.

[0190] The calculation unit 223 is used to calculate the verification value based on the number of repeated pixels, the number of pixels in the target organ region in the third mask image, and the number of pixels in the target organ region in the fourth mask image.

[0191] The verification unit 224 is used to determine that the registered image verification is passed when the verification value is greater than a preset threshold, and to determine that the registered image verification is failed when the verification value is not greater than the preset threshold.

[0192] Specific limitations regarding the organ registration device can be found in the limitations of the organ registration method described above, and will not be repeated here. Specific limitations regarding the organ registration verification device can also be found in the limitations of the organ registration verification method described above, and will not be repeated here. The various modules in the aforementioned organ registration device and organ registration verification device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in hardware or independently of the processor in a computer device, or stored in software in the memory of a computer device, so that the processor can call and execute the operations corresponding to each module.

[0193] The tissue and organ registration method and the tissue and organ registration verification method provided in this application can be applied to, for example... Figure 19 The computer device shown can be a server or a terminal, and its internal structure diagram can be as follows. Figure 19As shown, the computer device includes a processor, memory, network interface, display screen, and input devices connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage media. The network interface is used to communicate with external terminals via a network connection. When executed by the processor, the computer program implements a method for registering and verifying tissues and organs. The display screen can be an LCD screen or an e-ink display screen. The input devices can be a touch layer covering the display screen, buttons, a trackball, or a touchpad mounted on the computer device casing, or an external keyboard, touchpad, or mouse.

[0194] Those skilled in the art will understand that Figure 19 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0195] In one embodiment, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to perform the following steps:

[0196] Acquire a first processed image of a first image and a second processed image of a second image; the first image and the second image contain the target organ of the same object at different times;

[0197] The target organ in the first processed image and the target organ in the second processed image are rigidly registered to obtain a rigid registration result;

[0198] The target organ in the rigid registration result and the target organ in the second processed image are elastically registered to obtain the elastic registration result.

[0199] The first image is processed based on the rigid registration result and the elastic registration result to obtain the registered image.

[0200] The computer device provided in the above embodiments has a similar implementation principle and technical effect to the above method embodiments, and will not be described again here.

[0201] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, the computer program performing the following steps when executed by a processor:

[0202] Acquire a first processed image of a first image and a second processed image of a second image; the first image and the second image contain the target organ of the same object at different times;

[0203] The target organ in the first processed image and the target organ in the second processed image are rigidly registered to obtain a rigid registration result;

[0204] The target organ in the rigid registration result and the target organ in the second processed image are elastically registered to obtain the elastic registration result.

[0205] The first image is processed based on the rigid registration result and the elastic registration result to obtain the registered image.

[0206] The computer-readable storage medium provided in the above embodiments has a similar implementation principle and technical effect to the above method embodiments, and will not be described again here.

[0207] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the methods described above. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, or optical storage, etc. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM), etc.

[0208] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0209] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the invention patent. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this patent application should be determined by the appended claims.

Claims

1. A method for registering tissues and organs, characterized in that, The method includes: Acquire a third image, which is an enhanced image; The third image is transformed to obtain the second image; the second image is an intraoperative plain scan image. A first processed image of a first image and a second processed image of a second image are acquired; the first image and the second image contain the same target organ of the same object at different times; the first image is a preoperative enhancement type image; The target organ in the first processed image and the target organ in the second processed image are rigidly registered to obtain a rigid registration result; The target organ in the rigid registration result and the target organ in the second processed image are elastically registered to obtain the elastic registration result. The first image is processed according to the rigid registration result and the elastic registration result to obtain a registered image; the registered image contains the feature information of the first image; Extract the target organ and internal detail mask from the third image to obtain a third mask image corresponding to the third image; Extract the target organ and internal detail mask from the registered image to obtain a fourth mask image corresponding to the registered image; Calculate the number of pixels that overlap between the target organ region in the third mask image and the target organ region in the fourth mask image; The verification value is obtained based on the number of repeated pixels, the number of pixels in the target organ region in the third mask image, and the number of pixels in the target organ region in the fourth mask image. If the verification value is greater than a preset threshold, the registered image is determined to have passed verification; if the verification value is not greater than the preset threshold, the registered image is determined to have failed verification.

2. The method according to claim 1, characterized in that, The step of processing the first image based on the rigid registration result and the elastic registration result to obtain the registered image includes: A mechanical model of the target organ is constructed based on the rigid registration results. The difference between the elastic registration result and the rigid registration result is used as the boundary condition to solve the mechanical model, thereby obtaining the deformation field of each point inside the target organ; The first image is processed based on the rigid transformation matrix in the rigid registration result and the deformation field of each point inside the target organ to obtain the registered image.

3. The method according to claim 2, characterized in that, The step of processing the first image based on the rigid transformation matrix in the rigid registration result and the deformation field of each point inside the target organ to obtain the registered image includes: Based on the rigid transformation matrix, the first image is translated and / or rotated to obtain the transformed first image; Based on the deformation field of each point inside the target organ, each point inside the target organ in the transformed first image is translated and / or rotated to obtain the registered image.

4. The method according to claim 3, characterized in that, The method further includes: The deformation field of each point inside the target organ is interpolated to obtain the deformation field of each pixel inside the target organ. The process of translating points within the target organ in the transformed first image based on the deformation field of each point inside the target organ to obtain the registered image includes: Based on the deformation field of each pixel inside the target organ, each pixel inside the target organ in the transformed first image is translated to obtain the registered image.

5. The method according to any one of claims 1-4, characterized in that, The process of acquiring the first processed image of the first image and the second processed image of the second image includes: Obtain the first image and the second image; Extract a first mask image of the target organ from the first image, and extract a second mask image of the target organ from the second image; The target organ in the first mask image is divided into a surface mesh to obtain the first processed image, and the target organ in the second mask image is divided into a surface mesh to obtain the second processed image.

6. The method according to claim 1, characterized in that, The step of performing image transformation on the third image to obtain the second image includes: The third image is processed to remove highlight information, thus obtaining the second image; Alternatively, the third image can be input into a preset image transformation network to obtain the second image.

7. The method according to any one of claims 1-4, characterized in that, The step of obtaining the verification value based on the number of repeated pixels, the number of pixels in the target organ region of the third mask image, and the number of pixels in the target organ region of the fourth mask image includes: The verification value is calculated using a preset formula based on the number of repeated pixels, the number of pixels in the target organ region of the third mask image, and the number of pixels in the target organ region of the fourth mask image.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 7.

Images