A medical image processing display method and system

Abstract

This invention provides a medical image processing and display method and system. The method includes the following steps: acquiring a medical modal image set corresponding to the acquired object; establishing a first three-dimensional reconstruction model using the medical modal image set to generate an initial image; displaying the initial image as a three-dimensional initial image for human eye observation using a light field display; determining the observer's target observation point on the three-dimensional initial image; determining the target object corresponding to the position of the target observation point, wherein the target object is a local tissue in the acquired object; retrieving a multi-view two-dimensional image set of the target object; generating a target image from the multi-view two-dimensional image set; and displaying the target image as a three-dimensional target image for human eye observation using the light field display. During the observation process, the observer can observe the target object from multiple angles, in a three-dimensional and intuitive manner without any manual operation.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of medical images, and in particular to a medical image processing and display method and system. BACKGROUND

[0002] Three-dimensional images have been widely used in the medical field. Doctors can determine possible problems of patients according to three-dimensional images, and can also observe three-dimensional images to plan or perform surgery on patients. In the medical field, any angle information of the organ or bone to be measured is particularly important. At present, most medical three-dimensional images are tomographic images or three-dimensional reconstruction images. By selecting a target layer image or adjusting the angle of a three-dimensional reconstruction image, image information can be observed. However, the essence is still presented through a 2D flat display, which has limitations for medical personnel who are not very good at spatial sense. Moreover, doctors need to constantly manually select the required image or manually adjust the angle of the three-dimensional reconstruction image and zoom in on the required image during actual surgery, which not only reduces the efficiency of surgery, but also easily leads to surgical errors. SUMMARY

[0003] Therefore, the present application aims to solve the technical problem of manually selecting, adjusting or zooming in on the required medical image in the prior art, and to provide a medical image processing and display method and system.

[0004] According to a first aspect, an embodiment of the present application provides a medical image processing and display method, comprising the following steps:

[0005] Obtaining a medical modality image set corresponding to a collection object, establishing a first three-dimensional reconstruction model using the medical modality image set, generating an initial image, and displaying the initial image as a three-dimensional initial image for human eye observation through a light field display;

[0006] Determining a target observation point of an observer on the three-dimensional initial image;

[0007] Determining a target object corresponding to the position of the target observation point, the target object being a local tissue in the collection object;

[0008] Retrieving a multi-view two-dimensional image set of the target object, generating a target image from the multi-view two-dimensional image set, and displaying the target image as a three-dimensional target image for human eye observation through the light field display.

[0009] Optionally, the determination of the target observation point of the observer on the three-dimensional initial image comprises:

[0010] Identifying the left eye pupil and the right eye pupil of the observer, and determining the midpoint between the left eye pupil and the right eye pupil;

[0011] The center point of the initial three-dimensional image is determined, and the first intersection point of the line connecting the center point and the midpoint with the initial three-dimensional image is taken as the target observation point.

[0012] Optionally, determining the target object corresponding to the location of the target observation point includes:

[0013] Based on the location coordinates of the target observation point, match the target object containing the location coordinates.

[0014] Optionally, the step of establishing a first three-dimensional reconstruction model using the medical modal image set and generating an initial image includes:

[0015] Using the aforementioned medical modal image set, a three-dimensional volume rendering was performed using a ray projection method to establish a first three-dimensional reconstruction model;

[0016] Acquire two-dimensional images of the first three-dimensional reconstruction model from different angles according to the preset acquisition direction and preset acquisition angle;

[0017] Two-dimensional images acquired from different angles are rendered separately, and the rendered two-dimensional images are automatically used to generate the initial image for display on the light field display.

[0018] Optionally, after determining the target object corresponding to the location of the target observation point, the method further includes:

[0019] The target object is preprocessed;

[0020] The preprocessed target object is reconstructed in three dimensions again to generate a second three-dimensional reconstruction model.

[0021] Two-dimensional images of the second three-dimensional reconstruction model are acquired from multiple angles to generate the multi-view two-dimensional image set.

[0022] Optionally, the preprocessing of the target object includes:

[0023] Rendering processing;

[0024] The process involves refining the analysis of the voxel information of the target object in the first 3D reconstruction model and adjusting the window width and window level of the target object.

[0025] Optionally, after establishing the first three-dimensional reconstruction model, the method further includes:

[0026] The tissues in the first three-dimensional reconstruction model are segmented separately;

[0027] Each segmented tissue will generate a corresponding second three-dimensional reconstruction model;

[0028] For each of the second three-dimensional reconstruction models, multi-angle acquisition is performed to generate corresponding multi-view two-dimensional image sets.

[0029] According to a second aspect, embodiments of the present invention provide a medical imaging system, comprising:

[0030] A motion-sensing interactive device used to identify the observer's left and right pupils;

[0031] An intelligent image analysis workstation, electrically connected to the somatosensory interaction device, is used to acquire a medical modal image set corresponding to the acquired object, establish a first three-dimensional reconstruction model using the medical modal image set, generate an initial image, determine the target observation point of the observer on the three-dimensional initial image, determine the target object corresponding to the position of the target observation point, retrieve a multi-view two-dimensional image set of the target object, and generate a target image from the multi-view two-dimensional image set.

[0032] A light field display, electrically connected to the image intelligent analysis workstation, is used to display the initial image or target image as a three-dimensional initial image for human visual observation;

[0033] A flat panel display, electrically connected to the image intelligent analysis workstation, is used to display the medical modal image set and the multi-view two-dimensional image set.

[0034] According to a third aspect, embodiments of the present invention provide a computer device, including: a memory and a processor, wherein the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the above-described medical image processing and display method by executing the computer instructions.

[0035] According to a fourth aspect, embodiments of the present invention provide a computer-readable storage medium storing computer instructions for causing a computer to perform the above-described medical image processing and display method.

[0036] The technical solution of this invention has the following advantages:

[0037] In this embodiment of the invention, a first three-dimensional reconstruction model is established using the acquired medical modal image set of the target object, and an initial image is generated. This initial image is then displayed as a three-dimensional initial image for human visual observation via a light field display. Next, based on the determined target observation point, the target object is identified. A multi-view two-dimensional image set corresponding to the target object is retrieved, and the multi-view two-dimensional image set is then used to generate a target image. This target image is then displayed as a three-dimensional target image for human visual observation via a light field display. In this embodiment, during the actual observation process, the observer does not need to manually adjust the image. The target object that the observer focuses on observing is determined based on the target observation point, and the initially displayed three-dimensional initial image is switched to the three-dimensional target image. During the observation process, the observer can observe the target object from multiple angles, in a three-dimensional and intuitive manner. Furthermore, the three-dimensional target image is a further display of the target object; the target image generated based on the initial image has a local magnification effect, highlighting the details of the tissue structure. Attached Figure Description

[0038] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0039] Figure 1 This is a flowchart illustrating a specific example of a medical image processing and display method according to Embodiment 1 of this application;

[0040] Figure 2 This is a schematic diagram illustrating a specific example of viewing a three-dimensional image displayed on a light field display from different perspectives in Embodiment 1 of this application;

[0041] Figure 3 This is a schematic diagram illustrating a specific example of determining a target observation point in Embodiment 1 of this application;

[0042] Figure 4 This is a schematic diagram illustrating a specific example of the multi-view image acquisition direction in Embodiment 1 of this application;

[0043] Figure 5 This is a schematic diagram of a specific example of a medical imaging system according to Embodiment 2 of this application;

[0044] Figure 6 This is a schematic diagram of a specific example of a computer device in Embodiment 3 of this application. Detailed Implementation

[0045] The technical solution of the present invention will now be clearly and completely described with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0046] In the description of this invention, it should be noted that the terms "center," "upper," "lower," "left," "right," "vertical," "horizontal," "inner," and "outer," etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are used only for the convenience of describing the invention and for simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on the invention. Furthermore, the terms "first," "second," and "third" are used for descriptive purposes only and should not be construed as indicating or implying relative importance.

[0047] In the description of this invention, it should be noted that, unless otherwise explicitly specified and limited, the terms "installation," "connection," and "linking" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; they can also refer to the internal connection of two components; and they can refer to a wireless connection or a wired connection. Those skilled in the art can understand the specific meaning of the above terms in this invention based on the specific circumstances.

[0048] Furthermore, the technical features involved in the different embodiments of the present invention described below can be combined with each other as long as they do not conflict with each other.

[0049] Example 1

[0050] This embodiment provides a medical image processing and display method. This method involves a motion-sensing interactive device detecting the specific position of the observer's pupil, an image intelligent analysis workstation performing 3D reconstruction and target object identification on the acquired medical modal image set, and then a light field display showing the processed initial or target image as a 3D image for human visual observation, thereby realizing the processing and display of medical images. Figure 1 As shown, it includes the following steps:

[0051] Step S101: Obtain the medical modal image set corresponding to the acquisition object, establish a first three-dimensional reconstruction model using the medical modal image set, generate an initial image, and display the initial image as a three-dimensional initial image for human eye observation through a light field display.

[0052] Medical modal imaging can be multimodal medical imaging, such as CT modal medical imaging (Computed Tomography, CT), MR modal medical imaging (Nuclear Magnetic Resonance, MR), PET modal medical imaging (Positron Emission Tomography, PET), etc. All medical modal images of the acquired object are integrated into a medical modal image set, which can correspond to one modal medical image or multiple modal medical images.

[0053] Taking a medical modality image set containing only one type of medical modality image as an example, this embodiment can use a volume rendering method based on ray projection to visualize and render the images in the medical modality image set, forming a volume rendering model. This volume rendering model is used as the first 3D reconstruction model in this embodiment. Further, the first 3D reconstruction model is reconstructed into multiple viewpoints, for example, 100 viewpoints, with a 3.6° interval between each pair of viewpoints. A virtual camera is set on each viewpoint, with the camera center aligned with the center of the first 3D reconstruction model. Each viewpoint forms an image, which is then rendered to generate a 2D rendering image, and all rendering images are output simultaneously. The images rendered from each viewpoint are arranged in sequence and input to a light field display. The light field display automatically generates an initial image and displays it as a 3D initial image for human observation. In this embodiment, the initial image can also be generated by an image intelligent analysis workstation and then input to the light field display, which displays the initial image as a 3D initial image for human observation.

[0054] Step S102: Determine the target observation point of the observer on the initial three-dimensional image.

[0055] In this embodiment, a motion-sensing interactive device can first be used to detect the observer's left and right pupils, analyze the central viewing angle of the observer's left and right pupils, that is, the observation angle, automatically detect the observer's key observation position when observing the three-dimensional initial image, and determine the observer's target observation point on the three-dimensional initial image. The specific determination method will be introduced below.

[0056] It should be noted that the light field display used in this embodiment is a display device that utilizes an optical system to allow an observer to view rendered images from different angles, and can simultaneously display images from multiple viewpoints. For example... Figure 2The image shown illustrates the view of a 3D image displayed on a light field display from different perspectives. This display device utilizes optical principles to display a processed initial image or target image as a 3D initial image or 3D target image. A point on the 3D initial image can be mapped to the same point on the initial image; therefore, a target observation point on the 3D initial image can be mapped to a target observation point on the initial image.

[0057] Step S103: Determine the target object corresponding to the location of the target observation point, wherein the target object is a local tissue in the collected object.

[0058] After identifying the target observation point, the image intelligent analysis workstation automatically retrieves the three-dimensional coordinates of the target observation point in the coordinate system. Each tissue in the initial three-dimensional image displayed on the light field display for human visual observation corresponds to a set of three-dimensional coordinates. Based on the three-dimensional coordinates corresponding to the identified target observation point, the workstation automatically matches the target object containing those coordinates. For example, suppose the initial three-dimensional image contains two tissues: the liver and a blood vessel 1 connecting to the liver. The three-dimensional coordinate set corresponding to the liver is {(a1,b1,c1),(a2,b2,c2),(a3,b3,c3)}, and the three-dimensional coordinate set corresponding to blood vessel 1 is {(a11,b11,c11),(a22,b22,c22),(a33,b33,c33)}. If the three-dimensional coordinates corresponding to the identified target observation point are (a2,b2,c2), then the target object corresponding to the identified target observation point is the liver.

[0059] Step S104: Retrieve the multi-view two-dimensional image set of the target object, generate a target image from the multi-view two-dimensional image set, and display the target image as a three-dimensional target image for human eye observation through the light field display.

[0060] After identifying the target object, a third-dimensional reconstruction is performed to generate a volume rendering model. This volume rendering model serves as the second 3D reconstruction model in this embodiment. Further, the second 3D reconstruction model is reconstructed into multiple viewpoints, and images corresponding to each viewpoint are acquired. Each image is rendered to generate a 2D rendered image, and all rendered images are output simultaneously. The rendered images from each viewpoint are arranged in order to generate a multi-view 2D image set of the target object, which is then input to a light field display. The light field display automatically generates the target image and displays it as a 3D target image for human visual observation.

[0061] In this embodiment, the multi-view two-dimensional image set corresponding to the target object can also be directly retrieved. After establishing the first three-dimensional reconstruction model, the tissues in the first three-dimensional reconstruction model can be segmented, and a corresponding second three-dimensional reconstruction model can be generated for each segmented tissue. Multi-angle acquisition can be performed on each second three-dimensional reconstruction model to generate a corresponding multi-view two-dimensional image set. Once the target object is determined, the multi-view two-dimensional image set corresponding to the target object can be directly retrieved.

[0062] In this embodiment, a first 3D reconstruction model is established using the acquired medical modal image set of the target object, and an initial image is generated. This initial image is then displayed as a 3D initial image for human visual observation via a light field display. Next, based on the determined target observation point, the target object is identified. A multi-view 2D image set corresponding to the target object is retrieved, and the multi-view 2D image set is used to generate a target image. This target image is then displayed as a 3D target image for human visual observation via a light field display. In this embodiment, during the actual observation process, the observer does not need to manually adjust the image. The target object being observed is determined based on the observer's target observation point, and the initially displayed 3D initial image is switched to the 3D target image. During the observation process, the observer can observe the target object from multiple angles, in a three-dimensional and intuitive manner, without wearing any equipment. The 3D target image is a further display of the target object; that is, the second 3D reconstruction model is a 3D model reconstructed based on the target object. The regenerated target image, based on the initial image, has a local magnification effect, highlighting the details of the tissue structure and improving the display effect.

[0063] As an optional implementation, in this embodiment of the invention, determining the target observation point of the observer on the three-dimensional initial image includes:

[0064] Identify the observer's left and right pupils and determine the midpoint between the left and right pupils;

[0065] The center point of the initial three-dimensional image is determined, and the first intersection point of the line connecting the center point and the midpoint with the initial three-dimensional image is taken as the target observation point.

[0066] like Figure 3 As shown, firstly, the observer's left pupil L and right pupil R are identified using a motion-sensing interaction device, and the midpoint Q of the line connecting the left pupil L and the right pupil R is analyzed. Given the center point O of the initial 3D image, the first intersection point P of the line connecting the center point O and the midpoint Q with the initial 3D image is calculated, and this first intersection point is taken as the target observation point.

[0067] The specific calculation method can be as follows: taking the center point O of the initial three-dimensional image as the point of the three-dimensional coordinate system, using a depth camera to detect the positions of the observer's left pupil L and right pupil R in the three-dimensional coordinate system, and calculating the three-dimensional coordinates of point Q, establishing a relationship function for the OQ line, calculating all coordinates in the initial three-dimensional image that satisfy the relationship function based on the relationship function, and finally selecting the coordinate closest to the observer from all the coordinates that satisfy the condition as the first intersection point with the initial three-dimensional image, and taking this intersection point as the target observation point.

[0068] In this embodiment, the target observation point is determined by utilizing the positional relationship between the midpoint of the line connecting the left and right pupils of the observer and the center point of the initial three-dimensional image. This allows for the rapid identification of the observer's key observation location. Based on the determined target observation point, the target object observed by the observer can be further quickly identified and used by the light field display to further display the three-dimensional target image for human eye observation, thus achieving the effect of highlighting the observer's key observation location.

[0069] As an optional implementation, in this embodiment of the invention, determining the target object corresponding to the location of the target observation point includes:

[0070] Based on the location coordinates of the target observation point, match the target object containing the location coordinates.

[0071] Specifically, the first 3D reconstruction model established based on the medical modal image set of the acquired object can include multiple different tissue structures, such as organs and blood vessels. In the first 3D reconstruction model, each tissue is independently divided, and each tissue corresponds to a set of 3D coordinates. In this embodiment, the tissue corresponding to the target observation point, i.e., the target object, can be quickly matched based on the position coordinates of that position coordinate.

[0072] In this embodiment, coordinates are established when the first three-dimensional reconstruction model is built, and the three-dimensional coordinate set corresponding to each tissue is determined. When the observer actually observes, the target object can be quickly matched, reducing the operating load of the image intelligent analysis workstation. Moreover, the matching speed is fast, making it easy for the observer to quickly determine the required tissue structure.

[0073] In this embodiment, the identified target object can also be highlighted to facilitate the observer's quick location of the tissue.

[0074] In this embodiment, after determining the target object using a location coordinate matching method, an image segmentation method can be used to segment the target image. Specifically, the Otsu method (OSTU) based on thresholding can be used to segment the lung parenchyma, and post-processing of the segmentation results involving cavity filling and closure operations can be performed; methods based on histogram distribution and region growing can be used to segment the sternum, ribs, etc.; template-based segmentation methods can be used to segment the heart; and circular detection methods based on Hough transform and region growing methods based on seed points can be used to segment blood vessels and trachea. After segmentation, three-dimensional data of the thoracic tissue trachea model can be obtained; and the corresponding segmentation model or method is automatically called according to the tissue type of the target object or the location information contained in the target image to obtain the target object.

[0075] As an optional implementation, in this embodiment of the invention, the step of establishing a first three-dimensional reconstruction model using the medical modality image set and generating an initial image includes:

[0076] Using the aforementioned medical modal image set, a three-dimensional volume rendering was performed using a ray projection method to establish a first three-dimensional reconstruction model;

[0077] Acquire two-dimensional images of the first three-dimensional reconstruction model from different angles according to the preset acquisition direction and preset acquisition angle;

[0078] Two-dimensional images acquired from different angles are rendered separately, and the rendered two-dimensional images are input into the light field display according to the acquisition order to automatically generate an initial image for display.

[0079] Taking a medical modality image set containing only one type of medical modality image as an example, in this embodiment, a volume rendering method based on ray projection can be used to visualize and render the images in the medical modality image set to form a volume rendering model, which is then used as the first three-dimensional reconstruction model in this embodiment.

[0080] Furthermore, the first 3D reconstruction model is reconstructed into multiple viewpoints, and 2D images of the first 3D reconstruction model from different angles are acquired according to a preset acquisition direction and preset acquisition angle. For example, as... Figure 4As shown, three trajectory directions, G1, G2, and G3, can be used as preset acquisition directions. Each acquisition direction has a preset acquisition angle of 3.6°, meaning each viewpoint is spaced 3.6° apart. This allows the first 3D reconstruction model to be reconstructed into 300 viewpoints. A virtual camera is set up at each viewpoint, with the camera center aligned with the center of the first 3D reconstruction model. Each viewpoint forms an image, which is then rendered to generate a 2D rendering image. All rendering images are output simultaneously. The acquisition angle can be set according to requirements. The rendered images from each viewpoint are arranged in order according to the front-to-back direction of the acquired image and input to a light field display. The light field display automatically generates an initial image and displays it as a 3D initial image for human observation. In this embodiment, an image intelligent analysis workstation can also generate the initial image and then input it to the light field display, which displays it as a 3D initial image for human observation.

[0081] In this embodiment, after acquiring images from each viewpoint, each image is rendered to more closely approximate the shape and color of the organ or tissue itself, allowing the observer to observe more intuitively. Furthermore, for viewpoints where no virtual camera was set up, i.e., those not acquired, linear interpolation can be used to generate interpolated rendering images.

[0082] As an optional implementation, in this embodiment of the invention, after determining the target object corresponding to the location of the target observation point, the method further includes:

[0083] The target object is preprocessed;

[0084] The preprocessed target object is reconstructed in three dimensions again to generate a second three-dimensional reconstruction model.

[0085] Two-dimensional images of the second three-dimensional reconstruction model are acquired from multiple angles to generate the multi-view two-dimensional image set.

[0086] In this embodiment, the identified target object is reconstructed in three dimensions again based on the medical modality image set corresponding to the target object to generate a second three-dimensional reconstruction model. Similarly, a multi-angle acquisition method is used to generate a multi-view two-dimensional image set.

[0087] As an optional implementation, in this embodiment of the invention, the preprocessing of the target object includes:

[0088] The rendering process renders the target object into a form that more closely resembles the actual object's color or shape. It can also perform multi-view 2D rendering of the target object from the reverse angle and output the rendered image simultaneously with the image rendered from the forward view.

[0089] The process involves refining the analysis of the voxel information of the target object in the first 3D reconstruction model and adjusting the window width and window level of the target object.

[0090] In this embodiment, the preprocessing range can be set by offsetting the line connecting the midpoint Q and the center point O by ±5° in all directions, or preprocessing can be performed only on the target object. Based on the range or object to be preprocessed, the average and standard deviation of the voxel intensity values ​​are analyzed. Using one or more distribution functions such as Gaussian, uniform, and gamma distributions, the displayed intensity range and median are calculated and converted into window width and window level, achieving a more accurate and detailed display of the target object or range.

[0091] As an optional implementation, in this embodiment of the invention, after establishing the first three-dimensional reconstruction model, the method further includes:

[0092] The tissues in the first three-dimensional reconstruction model are segmented separately;

[0093] Each segmented tissue will generate a corresponding second three-dimensional reconstruction model;

[0094] For each of the second three-dimensional reconstruction models, multi-angle acquisition is performed to generate corresponding multi-view two-dimensional image sets.

[0095] In this embodiment, besides the method of further reconstructing the target object after determining it, another approach is to segment the tissues in the first three-dimensional reconstruction model after establishing the first three-dimensional reconstruction model, and generate a corresponding second three-dimensional reconstruction model for each segmented tissue. For example, the first three-dimensional reconstruction model contains three tissues: the liver and blood vessels 1 and 2 connecting the liver. The three tissues are segmented, and the segmented liver tissue generates a second three-dimensional reconstruction model for the liver, the segmented blood vessel 1 connecting the liver generates a second three-dimensional reconstruction model for blood vessel 1, and the segmented blood vessel 2 connecting the liver generates a second three-dimensional reconstruction model for blood vessel 2. Multi-angle acquisition is then performed on the second three-dimensional reconstruction models of the liver, blood vessel 1, and blood vessel 2 to generate multi-view two-dimensional image sets of the liver, blood vessel 1, and blood vessel 2, respectively.

[0096] When the observer actually observes, once the target object is identified, the corresponding multi-view two-dimensional image set can be directly retrieved, which reduces the operating load of the image intelligent analysis workstation. Moreover, the retrieval speed is fast, and it can be directly used to generate the target image. The speed of switching to the target image is also fast.

[0097] Example 2

[0098] This embodiment provides a medical imaging system that can be used to execute the medical image processing and display method in Embodiment 1 above, such as... Figure 5 As shown, the system includes:

[0099] The motion-sensing interaction device 11 is used to identify the observer's left and right pupils;

[0100] The image intelligent analysis workstation 12 is electrically connected to the somatosensory interaction device and is used to acquire a medical modal image set corresponding to the acquired object, establish a first three-dimensional reconstruction model using the medical modal image set, generate an initial image, determine the target observation point of the observer on the three-dimensional initial image, determine the target object corresponding to the position of the target observation point, retrieve a multi-view two-dimensional image set of the target object, and generate a target image from the multi-view two-dimensional image set.

[0101] The light field display 13 is electrically connected to the image intelligent analysis workstation and is used to display the initial image or target image as a three-dimensional initial image for human eye observation.

[0102] The flat panel display 14 is electrically connected to the image intelligent analysis workstation and is used to display the medical modal image set and the multi-view two-dimensional image set.

[0103] In this embodiment, the motion-sensing interaction device can use Kinect or Leapmotion to detect and track the pupil, and is equipped with a depth camera; the image intelligent analysis workstation can be a deep learning workstation based on the x86 architecture; the light field display can be LookingGlass; and the flat panel display can be a medical imaging display.

[0104] The medical imaging system may also include a planar interactive device 15, connected to a flat panel display 14, which allows for touch and interactive functions, such as adjusting the angle of the image on the flat panel display. Simultaneous display on the flat panel display and the light field display increases the observer's choice of observation options and allows for further confirmation of tissue details, thereby improving the accuracy of observation.

[0105] In this embodiment, a first 3D reconstruction model is established by acquiring a medical modal image set of the acquired object using an intelligent image analysis workstation, and an initial image is generated. This initial image is then displayed as a 3D initial image for human visual observation using a light field display. Next, based on the determined target observation point, the target object is identified. A multi-view 2D image set corresponding to the target object is retrieved, and the multi-view 2D image set is then used to generate a target image. This target image is then displayed as a 3D target image for human visual observation using a light field display. In this embodiment, during the actual observation process, the observer does not need to manually adjust the image. The target object that the observer focuses on can be determined based on the target observation point, and the initially displayed 3D initial image is switched to the 3D target image. During the observation process, the observer can observe the target object from multiple angles, in a three-dimensional and intuitive manner. Furthermore, the second 3D reconstruction model is a 3D model reconstructed based on the target object. The regenerated target image, based on the initial image, has a local magnification effect, highlighting the details of the tissue structure.

[0106] Example 3

[0107] This embodiment provides a computer device, such as... Figure 6 As shown, the computer device includes a processor 301 and a memory 302, wherein the processor 301 and the memory 302 can be connected via a bus or other means. Figure 6 Taking the example of a connection between China and Israel via a bus.

[0108] Processor 301 can be a Central Processing Unit (CPU). Processor 301 can also be other general-purpose processors, digital signal processors (DSPs), graphics processing units (GPUs), embedded neural network processing units (NPUs), or other dedicated deep learning coprocessors, application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, or combinations of the above types of chips.

[0109] The memory 302, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs, non-transitory computer-executable programs, and modules, such as the medical image processing and display method in this embodiment of the invention. Corresponding program instructions / modules are also stored. The processor 301 executes various functional applications and data processing by running the non-transitory software programs, instructions, and modules stored in the memory 302, thereby implementing the medical image processing and display method in the above embodiment.

[0110] The memory 302 may further include a program storage area and a data storage area, wherein the program storage area may store the operating system and applications required for at least one function; the data storage area may store data created by the processor 301, etc. Furthermore, the memory 302 may include high-speed random access memory and non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some embodiments, the memory 302 may optionally include memory remotely located relative to the processor 301, and these remote memories may be connected to the processor 301 via a network. Embodiments of the aforementioned network include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0111] The memory 302 stores one or more modules, which, when executed by the processor 301, perform actions such as... Figure 1 The medical image processing and display method in the illustrated embodiment.

[0112] For specific details regarding the aforementioned computer equipment, please refer to the relevant documentation. Figure 1 The relevant descriptions and effects in the illustrated embodiments are for understanding purposes only and will not be repeated here.

[0113] This invention also provides a computer-readable storage medium storing computer-executable instructions that can execute the medical image processing and display method in any of the above embodiments. The storage medium may be a magnetic disk, optical disk, read-only memory (ROM), random access memory (RAM), flash memory, hard disk drive (HDD), or solid-state drive (SSD), etc.; the storage medium may also include combinations of the above types of memory.

[0114] Obviously, the above embodiments are merely illustrative examples for clear explanation and are not intended to limit the implementation. Those skilled in the art will recognize that other variations or modifications can be made based on the above description. It is neither necessary nor possible to exhaustively list all possible implementations here. However, obvious variations or modifications derived therefrom are still within the scope of protection of this invention.

Claims

1. A method for processing and displaying medical images, characterized in that, include: The process involves: acquiring a medical modal image set corresponding to the target object; establishing a first three-dimensional reconstruction model using the medical modal image set; generating an initial image; and displaying the initial image as a three-dimensional initial image for human visual observation using a light field display. This includes: using the medical modal image set, performing three-dimensional volume rendering using a ray projection method to establish the first three-dimensional reconstruction model; acquiring two-dimensional images of the first three-dimensional reconstruction model at different angles according to a preset acquisition direction and angle; rendering the acquired two-dimensional images at different angles; and automatically generating an initial image for display on a light field display from the rendered two-dimensional images. The tissues in the first three-dimensional reconstruction model are segmented separately; Each segmented tissue will generate a corresponding second three-dimensional reconstruction model; For each of the second three-dimensional reconstruction models, multi-angle acquisition is performed to generate corresponding multi-view two-dimensional image sets; Determine the target observation point for the observer on the initial three-dimensional image; Determine the target object corresponding to the location of the target observation point, wherein the target object is a local tissue in the collected object; The multi-view two-dimensional image set of the target object is retrieved, the multi-view two-dimensional image set is used to generate a target image, and the target image is displayed as a three-dimensional target image for human eye observation through the light field display.

2. The medical image processing and display method according to claim 1, characterized in that, Determining the target observation point of the observer on the initial three-dimensional image includes: Identify the observer's left and right pupils and determine the midpoint between the left and right pupils; The center point of the initial three-dimensional image is determined, and the first intersection point of the line connecting the center point and the midpoint with the initial three-dimensional image is taken as the target observation point.

3. The medical image processing and display method according to claim 2, characterized in that, Determining the target object corresponding to the location of the target observation point includes: Based on the location coordinates of the target observation point, match the target object containing the location coordinates.

4. The medical image processing and display method according to claim 1, characterized in that, After determining the target object corresponding to the location of the target observation point, the method further includes: The target object is preprocessed; The preprocessed target object is reconstructed in three dimensions again to generate a second three-dimensional reconstruction model. Two-dimensional images of the second three-dimensional reconstruction model are acquired from multiple angles to generate the multi-view two-dimensional image set.

5. The medical image processing and display method according to claim 4, characterized in that, The preprocessing of the target object includes: Rendering processing; The process involves refining the analysis of the voxel information of the target object in the first 3D reconstruction model and adjusting the window width and window level of the target object.

6. A medical imaging system, characterized in that, include: A motion-sensing interactive device used to identify the observer's left and right pupils; The image intelligent analysis workstation is electrically connected to the somatosensory interaction device and is used to acquire the medical modal image set corresponding to the acquisition object, establish a first three-dimensional reconstruction model using the medical modal image set, and generate an initial image. Determine the target observation point for the observer on the initial 3D image; Determine the target object corresponding to the location of the target observation point; Retrieve a multi-view two-dimensional image set of the target object and generate a target image from the multi-view two-dimensional image set; wherein, the process includes: using the medical modal image set, employing a ray projection method to perform three-dimensional volume rendering and establish a first three-dimensional reconstruction model; acquiring two-dimensional images of the first three-dimensional reconstruction model from different angles according to a preset acquisition direction and preset acquisition angle; rendering the acquired two-dimensional images from different angles respectively, and automatically generating an initial image for display on a light field display from the rendered two-dimensional images; The image intelligent analysis workstation is also used to segment the tissues in the first three-dimensional reconstruction model; generate a corresponding second three-dimensional reconstruction model for each segmented tissue; and perform multi-angle acquisition on each second three-dimensional reconstruction model to generate a corresponding multi-view two-dimensional image set. A light field display, electrically connected to the image intelligent analysis workstation, is used to display the initial image or target image as a three-dimensional initial image for human visual observation; A flat panel display, electrically connected to the image intelligent analysis workstation, is used to display the medical modal image set and the multi-view two-dimensional image set.

7. A computer device, characterized in that, include: The system includes a memory and a processor, which are communicatively connected to each other. The memory stores computer instructions, and the processor executes the computer instructions to perform the medical image processing and display method according to any one of claims 1-5.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions for causing the computer to perform the medical image processing and display method according to any one of claims 1-5.

Images