Medical imaging method, apparatus and system

Abstract

To provide a medical imaging method for confirming in real time the position of a medical accessory such as a puncture needle intervening a subject.SOLUTION: The medical imaging method includes: determining the position of a medical accessory 300 in a reference coordinate system on the basis of a captured image acquired by an auxiliary camera 200; reconstructing a medical image according to a scanning result from a medical device 100 such as computed tomography on the basis of the reference coordinate system; and displaying in real time the position of the medical accessory in the medical image.SELECTED DRAWING: Figure 3

Description

[Technical field]

[0001] FIELD OF THE DISCLOSURE Embodiments of the present application relate to the field of medical devices, and in particular to medical imaging methods, devices and systems. [Background technology]

[0002] During the interventional operation, the medical accessory needs to be inserted into the object at a correct position and angle to ensure that the medical accessory can accurately reach the target position inside the object, and since the operator cannot directly observe the internal structure of the object, the operator cannot check the position of the medical accessory inside the object in real time.

[0003] In order to confirm the position of the medical accessory inside the object during the interventional operation, the object usually needs to be scanned multiple times, and the operator needs to adjust the position of the medical accessory inside the object multiple times according to the scanning result each time, so that the medical accessory finally reaches the target position. Here, the adjustment of the position of the medical accessory inside the object each time mainly depends on the experience of the operator, so the accuracy and reliability of the interventional operation cannot be ensured, and since multiple scanning is required during the interventional operation, the whole interventional operation process has many steps and takes a long time.

[0004] It should be noted that the above introduction of the background art is merely provided for the purpose of clearly and completely describing the technical solution of the present application, and facilitating the understanding of those skilled in the art. Summary of the Invention

[0005] SUMMARY OF THE DISCLOSURE Embodiments of the present application provide a medical imaging method, apparatus and system.

[0006] According to one aspect of an embodiment of the present application, there is provided a method for medical imaging, the method comprising: determining a position of the medical accessory in a reference coordinate system based on the captured image acquired by the auxiliary camera; reconstructing a medical image from the scan results with reference to the reference coordinate system; and displaying in real time the location of the medical accessory in the medical image.

[0007] According to one aspect of an embodiment of the present application, there is provided an apparatus for medical imaging, the apparatus comprising: a determination unit for determining a position of the medical accessory in a reference coordinate system based on the captured images acquired by the auxiliary camera; a reconstruction unit for reconstructing a medical image from a scan result based on the reference coordinate system; and a display unit for displaying in real time the location of the medical accessory in the medical image.

[0008] According to one aspect of an embodiment of the present application, there is provided a medical imaging system, the system comprising: a processor for determining a position of the medical accessory in a reference coordinate system based on the captured images obtained by the auxiliary camera, and for reconstructing a medical image from the scan results with reference to the reference coordinate system; and a display for displaying in real time the location of the medical accessory in the medical image.

[0009] One of the beneficial effects of the embodiment of the present application is that the position of the medical accessory in a reference coordinate system is determined based on the captured image obtained by the auxiliary camera, and the medical image is reconstructed from the scanning result based on the reference coordinate system, so that the position of the medical accessory in the medical image can be displayed in real time, which can improve the convenience of operation, help simplify and shorten the positioning process of the medical accessory, and is convenient for the operator to quickly and accurately move the medical accessory to the target position.

[0010] With reference to the following description and drawings, specific embodiments of the present application are disclosed in detail, showing the manner in which the principles of the present application can be employed. It should be understood that the present application is not limited in scope. The present application includes many changes, modifications, and equivalents within the spirit and terms of the appended claims. [Brief description of the drawings]

[0011] The included drawings are used to provide a further understanding of the embodiments of the present application, and constitute a part of the specification to illustrate the embodiments of the present application and explain the principles of the present application together with the text description. The drawings in the following description are only some examples of the present application, and it is obvious to those skilled in the art that other embodiments can be obtained based on these drawings without creative labor. In the drawings,

[0012] [Figure 1] FIG. 1 is a schematic diagram of a CT imaging device according to an embodiment of the present application. [Diagram 2] FIG. 1 is a schematic diagram of a medical imaging method according to an embodiment of the present application. [Diagram 3] FIG. 1 is a schematic diagram of a medical imaging system according to an embodiment of the present application. [Figure 4] FIG. 2 is a schematic diagram of an embodiment of Example 201 of the present application. [Diagram 5] FIG. 4 is a schematic diagram of an embodiment of Example 401 of the present application. [Figure 6] FIG. 2 is a schematic diagram of a method for determining a first transformation matrix in an embodiment of the present application. [Figure 7] FIG. 2 is a schematic diagram of a calibration jig according to an embodiment of the present application. [Figure 8] FIG. 2 is a schematic diagram of a scanning coordinate system and a world coordinate system according to an embodiment of the present application. [Figure 9] FIG. 2 is a schematic diagram of a display image in an embodiment of the present application. [Figure 10] FIG. 2 is another schematic diagram of a display image according to an embodiment of the present application. [Figure 11] FIG. 2 is another schematic diagram of a display image according to an embodiment of the present application. [Figure 12] FIG. 2 is another schematic diagram of the medical imaging method according to the embodiment of the present application. [Figure 13] FIG. 1 is a schematic diagram of a medical imaging apparatus according to an embodiment of the present application. [Figure 14] FIG. 2 is a schematic diagram of a computer device according to an embodiment of the present application. [Figure 15]FIG. 1 is a schematic diagram of a medical imaging system according to an embodiment of the present application. [Figure 16] FIG. 1 is a schematic diagram of a CT system according to an embodiment of the present application. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0013] The foregoing and other features of the embodiments of the present application will become apparent from the following specification, taken in conjunction with the drawings. Although the specification and drawings specifically disclose certain embodiments of the present application, which represent some embodiments in which the principles of the embodiments of the present application may be employed, it should be understood that the present application is not limited to the described embodiments, but rather, the embodiments of the present application include all modifications, variations, and equivalents falling within the scope of the appended claims.

[0014] In the embodiments of the present application, the terms "first", "second", etc. are used to distinguish different elements from one another, but do not indicate the spatial or chronological order of these elements, and these elements should not be limited to these terms. The term "and / or" includes any and all combinations of one or more of the associated listed terms. Terms such as "comprises", "includes", "having", etc. refer to the presence of stated features, elements, elements or assemblies, but do not exclude the presence or addition of one or more other features, elements, elements or assemblies.

[0015] In the embodiments of the present application, the singular forms "a", "the", and the like should be understood in the broad sense as "a kind" or "a class" including the plural, and not limited to the meaning of "one", and the term "said" should be understood to include the singular and the plural, unless otherwise clearly indicated in the context. In addition, the term "according to" should be understood as "at least in part depending on ..." and the term "based on" should be understood as "at least in part based on ...", unless otherwise clearly indicated in the context.

[0016] Features described and / or illustrated for one embodiment may be used in the same or similar manner in one or more other embodiments, may be combined with features in other embodiments, or may replace features in other embodiments. The term "comprises" as used herein means the presence of a feature, whole, step, or assembly, but does not exclude the presence or addition of one or more other features, wholes, steps, or assemblies.

[0017] The devices for obtaining medical image data (e.g., scan results) described herein may be applicable to various medical imaging modes, including, but not limited to, computed tomography (CT) devices, magnetic resonance imaging (MRI) devices, ultrasound imaging (Ultrasound Imaging) devices, positron emission computed tomography (PET) devices, single photon emission computed tomography (SPECT) devices, PET / CT, PET / MR, or any other suitable medical imaging device.

[0018] For example, CT is a continuous cross-sectional scan of a part of a scanning target using X-rays, and the X-rays transmitted through the cross-section are received by a probe and converted into visible light, or the received photon signal is directly converted and then processed to form medical image data. FIG. 1 is a schematic diagram of a CT imaging device according to an embodiment of the present application, and shows a CT imaging device 100. As shown in FIG. 1, the CT imaging device 100 includes a scanning gantry 101 and a patient table 102, the scanning gantry 101 having an X-ray source 103, which projects an X-ray beam toward a detector assembly or collimator 104 on the opposite side of the scanning gantry 101. A detection target 105 lies on the patient table 102 and moves with the patient table 102 to a scanning gantry opening 106, and medical image data of the detection target 105 can be obtained by scanning the X-ray source 103. The present application is not limited to this, and the CT imaging device 100 may have another structure.

[0019] MRI is based on the principle of nuclear magnetic resonance of atomic nuclei, and forms an image by transmitting RF pulses to a scanning object and receiving electromagnetic signals emitted from the scanning object through reconstruction.

[0020] PET involves accelerating charged particles in a cyclotron and colliding them with a target nucleus, generating a radioactive nuclide with a positron through a nuclear reaction, synthesizing a developer and introducing it into the body to localize it in the target organ, releasing a positively charged electron during the decay process, and after traveling a very short distance through the tissue, these positrons interact with electrons in the surrounding material to generate annihilation radiation, releasing two photons with the same energy but in the opposite direction. PET imaging employs a series of paired transducers arranged at 180 degrees to each other and then connecting appropriate lines to detect the photons of annihilation radiation generated by the tracer outside the body, and the collected information is processed by a computer to obtain a reconstructed image.

[0021] SPECT uses a radioisotope as a tracer, which is injected into the human body and concentrated in the examined organ, making the organ a gamma ray source. A probe that rotates around the body outside the body records the distribution of radioactivity in the organ tissue. When the probe rotates one angle, one set of data is obtained, and when it rotates one cycle, several sets of data are obtained. A series of cross-sectional images can be created from these data, and the computer reconstructs the imaging in a cross-sectional manner.

[0022] PET and SPECT extend histopathological examination from the molecular level to tissue local biochemical display. The images they provide are images of the physiological metabolism of the human body, which are excellent in functional development and can detect the occurrence and development of disease, functional and metabolic changes. However, CT and MRI are excellent in accurately reflecting morphological and structural changes. In contrast, traditional methods can use CT or MRI to perform attenuation correction on PET or SPECT images. That is, by integrating PET or SPECT with CT or MRI, the complementarity of functional images and anatomical image information can be realized, and better differentiation and diagnosis can be achieved.

[0023] The system for obtaining medical image data may include the above-mentioned medical imaging device, and / or an independent computer device connected to the medical imaging device, and / or a computer device connected to the cloud of the Internet, which computer device is connectable to the medical imaging device or a memory for storing medical images by the Internet. The imaging method may be performed independently or in cooperation with the above-mentioned medical imaging device, the computer device connected to the medical imaging device, or the computer device connected to the cloud of the Internet.

[0024] Also, the medical imaging workstation may be located locally to the medical imaging device, i.e., the medical imaging workstation may be located near the medical imaging device, or the medical imaging workstation and the medical imaging device may be co-located in a scanning room, imaging department, or the same hospital, and the medical imaging cloud platform analysis system may be located remotely from the medical imaging device, e.g., in a cloud that communicates with the medical imaging device.

[0025] As an example, after a medical institution completes an imaging scan with a medical imaging device, the scanned data can be stored in a storage device, and a medical imaging workstation can directly read the scanned data and perform image processing with its processor. As another example, a medical image cloud platform analysis system can read the medical images in the storage device through remote communication and provide "Software as a Service" (SaaS). SaaS may exist between hospitals, between hospitals and imaging centers, or between hospitals and third-party online medical service providers.

[0026] In embodiments of the present application, the term "object" may be equivalently replaced with "scanned object," "scanning object," "scanned object," "detecting object," "detecting object," or "detected object," and may include any object that is imaged. The terms "pixel" and "voxel" are used interchangeably. The terms "layer surface" and "edge surface" are used interchangeably.

[0027] Hereinafter, examples of the present application will be described in detail with reference to the drawings.

[0028] An embodiment of the present application provides a method for medical imaging. FIG. 2 is a schematic diagram of the medical imaging method of the embodiment of the present application. As shown in FIG. 2, the medical imaging method includes: determining 201 a position of the medical accessory in a reference coordinate system based on an image captured by the auxiliary camera; reconstructing 202 a medical image from the scan with respect to the reference coordinate system; and displaying 203 the location of the medical accessory in the medical image in real time.

[0029] According to the above embodiment, the position of the medical accessory in the reference coordinate system is determined based on the captured image acquired by the auxiliary camera, and the medical image is reconstructed from the scanning result based on the reference coordinate system, so that the position of the medical accessory in the medical image can be displayed in real time, which can improve the convenience of operation, help simplify and shorten the positioning process of the medical accessory, and facilitate the operator to quickly and accurately move the medical accessory to the target position.

[0030] In some embodiments, the medical imaging method may be performed in a medical imaging system. FIG. 3 is a schematic diagram of a medical imaging system according to an embodiment of the present application. The medical imaging system may include a processor (not shown) and a display (not shown), and the processor may determine the position of the medical accessory in a reference coordinate system based on the captured image acquired by the auxiliary camera, and reconstruct a medical image from the scan result based on the reference coordinate system. The display may display the position of the medical accessory in the medical image in real time. As shown in FIG. 3, the medical imaging system may further include a CT imaging device 100 and an auxiliary camera 200, and the CT imaging device 100 may be used to obtain a scan result, the structure of which can refer to the above content, but the description will not be repeated here, and the auxiliary camera 200 may be used to obtain a captured image. The present application is not limited thereto, and the medical imaging system may include other medical imaging devices as described above.

[0031] In some embodiments, the auxiliary camera 200 may be various components having an imaging function. For example, the auxiliary camera may be a depth camera, a thermography camera, etc. The auxiliary camera 200 may capture an image of a subject (e.g., a medical accessory 300, a target 400, etc.) that appears within the imaging range of the auxiliary camera 200, and generate a captured image including the subject based on an imaging coordinate system (Xc, Yc, Zc).

[0032] In some embodiments, the origin Oc of the imaging coordinate system (Xc, Yc, Zc) may be located at the optical center of the auxiliary camera, the Zc axis may be parallel to the optical axis of the auxiliary camera, and the Xc axis and the Yc axis are perpendicular to each other and perpendicular to the Zc axis, but the present application is not limited thereto, and the imaging coordinate system may be set in other ways.

[0033] In some implementations, 201 may determine the position of the medical accessory in the imaging coordinate system based on the captured image, and then determine the position of the medical accessory in the reference coordinate system by coordinate transforming the position of the medical accessory in the imaging coordinate system. Figure 4 is a schematic diagram of an embodiment of 201 in the present application. As shown in Figure 4, 201 includes: determining 401 a position of the medical accessory in an imaging coordinate system from the captured image; determining 402 a position of the medical accessory in the reference coordinate system according to the position of the medical accessory in the imaging coordinate system and a first transformation matrix between the imaging coordinate system and the reference coordinate system.

[0034] In some embodiments, the medical accessory 300 may be various components used in an interventional procedure, for example, the medical accessory may be a needle or needle assembly, a guidewire or guidewire assembly, a catheter or catheter assembly, etc. The number of medical accessories may be one or more.

[0035] Taking the medical accessory 300 as an example of a needle assembly, the medical accessory 300 may include a needle and a marker. In 401, the position of the marker in the captured image can be determined to determine the position of the needle in the captured image. FIG. 5 is a schematic diagram of an embodiment of 401 in the embodiment of the present application. As shown in FIG. 5, 401 includes: 501: Recognizing a marker in a captured image and determining an extension direction and position information of the marker in an image capture coordinate system; and determining 502 a position of the medical accessory in an imaging coordinate system according to the extension direction and position information and the relative positional relationship between the marker and the puncture needle.

[0036] In 501, a specific method for recognizing a marker in a captured image and determining the extending direction and position information of the marker in an image capture coordinate system can be found in the related art, but the description will not be repeated here.

[0037] In some embodiments, the relative positional relationship between the marker and the puncture needle (e.g., the angle between the extension directions of the two and / or the distance between the two) may be stored in advance, so that after obtaining the extension direction and position information of the marker, the extension direction and position information of the puncture needle can be determined based on the known relative positional relationship, thereby determining the position of the medical accessory in the imaging coordinate system.

[0038] In some embodiments, the extension direction of the marker and the extension direction of the puncture needle may be at a predetermined angle, for example, the marker may be parallel or perpendicular to the puncture needle.

[0039] In some embodiments, the marker may be fixed at a predetermined position on the needle, such that the distance between the marker and each position of the needle (eg, the tip) can be predetermined.

[0040] The present application is not limited to this, and the marker may carry information about the medical accessory, for example, the marker may be a graphic to which information can be attached, and by recognizing the marker, information indicating the relative positional relationship between the marker and the puncture needle can be obtained, thereby eliminating the need to store the relative positional relationship in advance.

[0041] In some embodiments, the marker may include at least one of a member having a predetermined temperature, a member having a predetermined shape and / or pattern, or a member that emits light according to a predetermined rule. By forming the marker in the above-mentioned form, the marker can be accurately and reliably recognized in the captured image. However, the present application is not limited thereto, and the marker may have other forms.

[0042] In some embodiments, the marker may be a sleeve fitted to the outer periphery of the puncture needle. The above-mentioned arrangement of the marker can ensure that the extension direction of the marker is parallel to the extension direction of the puncture needle, and is useful for determining the extension direction and position information of the puncture needle according to the extension direction and position information of the marker. The present application is not limited to this, and the marker may be of other types.

[0043] In some embodiments, the sleeve may be a separate reusable attachment, although the present application is not so limited, and the sleeve may be an integral part of the needle.

[0044] 5 is taken as an example to describe the embodiment of the embodiment 401 of the present application, but the present application is not limited to this, and the position of the medical accessory in the imaging coordinate system may be determined in other ways. For example, a deep learning algorithm is used to detect the medical accessory and its position from the camera depth and RGB data.

[0045] In some embodiments, the reference coordinate system may include a world coordinate system (X, Y, Z), although the present application is not so limited and the reference coordinate system may include other coordinate systems.

[0046] In some embodiments, the origin O of the world coordinate system (X, Y, Z) may be located at the center of a scanning gantry (e.g., the scanning gantry of the CT imaging device 100) as shown in Fig. 3. By placing the origin O of the world coordinate system (X, Y, Z) at the center of the scanning gantry, it is possible to simplify the process of reconstructing medical images (described later) and reduce the amount of calculation for coordinate transformation. However, the present application is not limited thereto, and the origin O of the world coordinate system (X, Y, Z) may be located at another position.

[0047] In some embodiments, one coordinate axis (e.g., X-axis) of the world coordinate system (X, Y, Z) may be parallel to the short side of the patient table 102, one coordinate axis (e.g., Z-axis) may be parallel to the long side of the patient table 102, and one coordinate axis (e.g., Y-axis) may be perpendicular to the patient table 102. By setting the coordinate axes of the world coordinate system in this way, it is possible to simplify the process of reconstructing medical images, which will be described later, and to reduce the amount of calculation for coordinate transformation. The present application is not limited to this, and the three coordinate axes of the world coordinate system (X, Y, Z) may be set in other forms.

[0048] In some embodiments, there may be various transformation relationships, e.g., affine transformations, between the imaging coordinate system (Xc, Yc, Zc) of the auxiliary camera and the world coordinate system (X, Y, Z), transforming a point (or position) in the imaging coordinate system to a point (or position) in the world coordinate system by rotation and / or scaling and / or translation.

[0049] In some embodiments, the transformation relationship between the imaging coordinate system (Xc, Yc, Zc) of the auxiliary camera and the world coordinate system (X, Y, Z) may be represented by a first transformation matrix. Here, the first transformation matrix may be determined in various ways. FIG. 6 is a schematic diagram of a method for determining the first transformation matrix. As shown in FIG. 6, the method for determining the first transformation matrix includes: Measure 601 a first calibration point coordinate in a world coordinate system of a calibration point on a calibration fixture placed on a surface of a stage (e.g., a patient table 102, etc.); 602: keeping the position of the calibration jig unchanged, acquiring a first captured image of the calibration jig, and determining second calibration point coordinates of the calibration point in the image coordinate system according to the first captured image of the calibration jig; and determining 603 a first transformation matrix according to the first calibration point coordinates and the second calibration point coordinates.

[0050] In some embodiments, the calibration jig may take a variety of forms. Figure 7 is a schematic diagram of a calibration jig in an embodiment of the present application. As shown in Figure 7, the calibration jig may be a flat plate having a black and white pattern, and the points where each black and white grid connects may be used as calibration points.

[0051] In some embodiments, the CT imaging device 100 may scan a subject and generate scan results, where the scan results may be represented in various forms, such as Digital Imaging and Communications in Medicine (DICOM) data, to facilitate data processing and exchange.

[0052] In some embodiments, the scan result may include multiple voxels, and the location of each voxel may be represented by coordinates (i, j, k) in a scan coordinate system (I, J, K). For example, the scan result may include multiple cross sections, and coordinate k may indicate the cross section in which the voxel is located, coordinate i may indicate the column in which the voxel is located in the cross section, and coordinate j may indicate the row in which the voxel is located in the cross section. This application is not limited in this respect, and coordinates (i, j, k) may have other meanings.

[0053] In some embodiments, a medical image is reconstructed from the scan results at 202. Figure 8 is a schematic diagram of a scan coordinate system (I, J, K) and a world coordinate system (X, Y, Z) in an embodiment of the present application. As shown in Figure 8, at 202, the position of the scan result in the scan coordinate system may be coordinate-transformed to obtain the position of the scan result in the reference coordinate system, i.e., the medical image may be reconstructed in the reference coordinate system.

[0054] In some embodiments, there may be various transformation relationships between the scan coordinate system (I, J, K) and the world coordinate system (X, Y, Z), e.g., affine transformations, which transform pixels in the scan coordinate system (I, J, K) to points (or positions) in the world coordinate system (X, Y, Z) by rotation and / or scaling and / or translation.

[0055] In some embodiments, the transformation relationship between the scan coordinate system (I, J, K) and the world coordinate system (X, Y, Z) may be represented by a second transformation matrix, where parameters in the second transformation matrix can be obtained from the scan result (e.g., DICOM data), such that each voxel can be mapped to the world coordinate system according to the scan result (e.g., DICOM data).

[0056] For example, take pixel (i, j, 0) in the scanning coordinate system as an example, and convert the pixel into coordinates (P x ,P y ,P z ) can be converted to

number

[0057] Here, (P x ,P y ,P z ) are the world coordinates of voxel (i, j) in the image plane, in millimeters (mm), and (S x ,S y ,S z ) is the image position (object) relative to the origin O, in mm, and (X x ,X y ,X z ) is the row (X) direction cosine value in the image direction (object), and (Y x ,Y y ,Y z) is the column (Y) direction cosine value in the image direction (object), i is the column index in the image plane, where the index of the first column is, for example, 0, Δi is the column pixel resolution in pixel pitch, in mm, and j is the row index in the image plane, where the index of the first row is, for example, 0, Δj is the row pixel resolution in pixel pitch, in mm. x ,S y ,S z ), (X x ,X y ,X z ), (Y x ,Y y ,Y z ), Δi and Δj) can be obtained from DICOM data, and for specific determination methods, reference can be made to related art, but the description will not be repeated here.

[0058] In some embodiments, the position of the object when the scan is generated and the position of the object when the intervention is performed may be different. For example, when the object is scanned, the object may be located inside the scanning gantry (e.g., a first position), and when the intervention is performed on the object, the object may be moved outside the scanning gantry (e.g., a second position). For example, the object may be held on the patient table 102 and when scanned, the object may be moved by the patient table 102 to the first position, and after the scan is completed, the object may be moved by the patient table 102 to the second position such that the intervention is performed on the object at the second position.

[0059] In the case of relative position information, in 202, a medical image may be reconstructed according to the relative positional relationship between the first position and the second position. For example, the position of the scan result in the scanning coordinate system is coordinate-transformed according to the second transformation matrix, and then the transformed coordinates are retransformed according to the above-mentioned relative positional relationship to obtain the position in the reference coordinate system.

[0060] Here, the relative positional relationship between the first position and the second position can be determined by moving the patient platform 102. The present application is not limited to this, and the relative positional relationship between the first position and the second position may be determined according to other methods. For example, at least three markers are placed on the surface of the object, and the relative positional relationship between the first position and the second position is determined according to the positions of the three markers when the object is scanned and the positions of the three markers when an intervention operation is performed. This allows the above-mentioned relative positional relationship to be accurately determined even if the object is displaced relative to the patient platform 102.

[0061] In some embodiments, when the position of the medical accessory in the reference coordinate system and the position of the medical image in the reference coordinate system are determined in 203, the position of the medical accessory in the medical image can be displayed in the reference coordinate system in real time according to the position information. For specific methods of displaying according to the position, reference can be made to the related art, and the description will not be repeated here.

[0062] In some embodiments, the location of the medical accessory may be the location of at least a part of the medical accessory. For example, when the medical accessory is a needle, the location may be the location of the tip of the needle, the location may be the location of the entire needle, etc.

[0063] In some embodiments, the medical image in the reference coordinate system may include a three-dimensional image, which is generated from a scanning result (e.g., DICOM data) by a three-dimensional modeling algorithm. FIG. 9 is a schematic diagram of a display image of an embodiment of the present application. As shown in FIG. 9, in 203, the position of the medical accessory 300 may be displayed in real time on a three-dimensional medical image 901. The medical image 901 may include an area 902 corresponding to the scanning result, so that the relative positional relationship between the medical accessory 300 and the area 902 can be displayed in real time. This helps the operator to timely adjust the angle and position of the medical accessory 300 according to the relative positional relationship.

[0064] In some embodiments, other auxiliary information may be displayed on the three-dimensional image of the medical image, for example, a planned movement path of the medical accessory, and / or information indicating a relative positional relationship between the medical accessory and a start point and / or a destination point of the movement path, and / or a predicted path determined by the position and extension direction of the medical accessory, and / or an imaged image may be superimposed on the three-dimensional image.

[0065] For example, after obtaining a scan result, a movement path of the medical accessory may be planned according to the scan result, and when an intervention operation is performed, the medical accessory may be moved according to the planned movement path, so that the safety and reliability of the intervention operation can be ensured. Here, the movement path may include a starting point, a target point, and a path between the starting point and the target point. The starting point may be, for example, the insertion point of the puncture needle, and the target point may be, for example, the target position to which the puncture needle reaches, such as the position of the lesion. By superimposing the movement path on the three-dimensional image, the operator can intuitively know whether the current position of the medical accessory is deviated from the movement path, and thus the angle and position of the medical accessory 300 can be adjusted in a timely manner.

[0066] Alternatively, information indicating the relative positional relationship between the medical accessory and the start point and / or target point of the movement path may be superimposed, and the information may include, for example, angle information and / or distance information of the tip of the puncture needle relative to the start point and / or target point, or may include an icon pointing from the tip of the puncture needle to the start point and / or target point, thereby helping the operator intuitively know the position of the medical accessory.

[0067] Alternatively, a predicted path determined by the position and extension direction of the medical accessory may be superimposed, for example shown as a line of a different color or shape from the icon of the medical accessory, thereby helping the operator know the position the medical accessory will reach when propelling the medical accessory according to its current position and angle.

[0068] Alternatively, an image captured by an auxiliary camera may be superimposed on the three-dimensional image. Here, the image captured by the auxiliary camera may include an object, and superimposing the image on the three-dimensional image helps the operator to know the relative positions of the medical accessory, the area 902, and the object.

[0069] In some embodiments, the medical images may include two-dimensional images displayed in one or more planes, where the two-dimensional images are generated from a scan (e.g., DICOM data) by a multi-planar reconstruction algorithm. For example, the two-dimensional images may include sagittal, coronal, and transverse planes. The application is not so limited, and the two-dimensional images may include planar images in other orientations.

[0070] In some embodiments, in addition to displaying the position of the medical accessory in the medical image in real time, a two-dimensional image corresponding to the position of the medical accessory may be displayed in real time according to the position of the medical accessory, which helps the operator to know the position of the medical accessory more clearly in the medical image.

[0071] In some embodiments, the two-dimensional image corresponding to the location of the medical accessory may include a first two-dimensional image passing through the location of the medical accessory and / or a second two-dimensional image passing through a target point of the medical accessory orthogonal to the first two-dimensional image passing through the location of the medical accessory, whereby the first two-dimensional image can provide the operator with information about tissues and / or structures near the location of the medical accessory, and the second two-dimensional image can provide the operator with information about tissues and / or structures near the location of the lesion, whereby helping the operator locate the location of the medical accessory and the target point.

[0072] In some embodiments, the two-dimensional images corresponding to the location of the medical accessory may include a first two-dimensional image passing through the location of the medical accessory and a third two-dimensional image passing through the location of the medical accessory and orthogonal to the first two-dimensional image, which may provide the operator with more detailed information regarding tissues and / or structures near the location of the medical accessory.

[0073] 10 is another schematic diagram of a display image according to an embodiment of the present application. As shown in FIG. 10, depending on the position of the medical accessory 300, for example, the tip position of the puncture needle, a sagittal plane 1001, a coronal plane 1002, and a transverse plane 1003 passing through the tip position of the puncture needle may be displayed.

[0074] In some embodiments, the first two-dimensional image and / or the second two-dimensional image and / or the third two-dimensional image may include mark information of a target point. The mark information may include information indicating the position of the target point and / or information regarding a relative positional relationship between the medical accessory and the target point. For example, the mark information may include a direction and a distance that the medical accessory needs to move from a current position to reach the target point, etc., which helps an operator to accurately adjust the position and angle of the medical accessory.

[0075] In some embodiments, as shown in FIG. 2, the method includes: generating 204 a grid matrix from the scan results, the grid matrix being marked with start and / or destination points of the planned medical accessory movement path; and superimposing 205 a grid matrix on the captured image.

[0076] This allows the position information of the start point and / or the target point to be provided to the operator in the captured image, which helps the operator intuitively know the positions of the start point and / or the target point from the captured image. In addition, there is no need to generate a grid matrix using some external physical assemblies, which simplifies the operation.

[0077] Fig. 11 is another schematic diagram of a display image in an embodiment of the present application. As shown in Fig. 11, a captured image 1100 may include a real image 1101 of an object 400, and a grid matrix 1102 is superimposed on the captured image 1100, and a starting point A is added to the grid matrix 1102, so that an operator can intuitively know the position of the starting point A on the object 400, which helps the operator quickly position the medical accessory 300 at the starting point A, and improves the efficiency and accuracy of the intervention operation.

[0078] Conventionally, in order to make it easier for an operator to determine the starting point A, it was usually necessary to add a wire mesh to the surface of the target 400, scan the target 400 with the wire mesh added, and obtain the scanning results. The position of the starting point A on the target 400 is determined based on the relative position information between the wire mesh image in the scanning results and the starting point A of the planned moving path. For example, it is determined from the scanning results that the starting point A is located within the grid of the nth row and mth column of the wire mesh, and the operator determines the position of the starting point A on the target 400 by finding the grid of the nth row and mth column in the wire mesh added to the surface of the target 400.

[0079] Compared to conventional techniques, the superimposition of a grid matrix on the captured image reduces the cost of interventional scanning by eliminating the need for additional disposable auxiliary consumables (e.g., wire mesh) and the need for additional hardware to assist in the positioning of medical accessories.

[0080] In some embodiments, the grid matrix 1102 may be generated from a scan result. For example, in the case of DICOM data, every pixel of the DICOM data may be mapped from the scan coordinate system to the world coordinate system, and every pixel may be mapped to the imaging coordinate system. In this way, the start point and / or the target point of the planned path in the scan coordinate system can be mapped to the imaging coordinate system to generate a grid matrix with the start point and / or the target point.

[0081] In some embodiments, the grid matrix may display information in real time indicating the position of the medical accessory and / or the relative position of the medical accessory to the start point and / or the target point. As shown in Fig. 11, an image of the medical accessory 300 may be displayed in a captured image 1100, which can provide the operator with a more intuitive view of the position of the medical accessory 300 relative to the start point A, and the operator can intuitively know the distance between the current position of the medical accessory 300 and the start point A using the grid matrix 1102 as a scale.

[0082] As shown in FIG. 11 , the information 1103 indicating the relative positional relationship between the medical accessory 300 and the start point and / or the target point may be an arrow or distance information pointing from the current position of the medical accessory 300 to the start point and / or the target point, which helps the operator quickly place the medical accessory 300 in the correct position.

[0083] 12 is another schematic diagram of a medical imaging method according to an embodiment of the present application. Hereinafter, with reference to FIG. 12, the medical imaging method according to an embodiment of the present application will be described by taking an example in which the object is a patient and the medical accessory includes a puncture needle assembly. As shown in FIG. 12, the medical imaging method includes: Scanning a patient and acquiring a diagnostic scan sequence 1201; determining puncture path 1202 by a diagnostic scan sequence; generating 1203 a multiplanar reconstruction (MPR) and 3D volumetric image model based on the diagnostic scan sequence; 1204, displaying the puncture path using the MPR and 3D volume image model; generating a grid matrix from the diagnostic scan sequence and marking 1205 locations of insertion points on the grid matrix; A grid matrix is ​​used to cover the image of the patient, and an arrow is displayed from the tip of the needle to the insertion point 1206; The auxiliary camera monitors the image of the puncture needle and the patient in real time so that the user places the puncture needle at the insertion point according to the grid of the grid matrix on the imaging screen 1207; The method may also include dynamically displaying the position of the puncture needle in the 3D volume image model according to the angle and position of the puncture needle, and dynamically displaying an MPR image corresponding to the angle and position of the puncture needle, as the user moves the puncture needle to the target point and completes the puncture.

[0084] It should be noted that the above figures 2, 4, 5, 6 and 12 are merely schematic illustrations of the embodiments of the present application, but the present application is not limited thereto. For example, the execution order between each operation can be appropriately adjusted, and some other operations can be added or some operations can be removed. Those skilled in the art can make appropriate modifications according to the above content, without being limited to the descriptions of the above figures 2, 4, 5, 6 and 12.

[0085] As can be seen from the above embodiments, the position of the medical accessory in the reference coordinate system is determined based on the captured image obtained by the auxiliary camera, and the medical image is reconstructed from the scanning result based on the reference coordinate system, so that the position of the medical accessory in the medical image can be displayed in real time, which can improve the convenience of operation, help simplify and shorten the positioning process of the medical accessory, and facilitate the operator to quickly and accurately move the medical accessory to the target position.

[0086] The embodiment of the present application further provides a medical imaging apparatus, and the description of the overlapping parts with the above-mentioned embodiment will not be repeated. Fig. 13 is a schematic diagram of the medical imaging apparatus of the embodiment of the present application. As shown in Fig. 13, the medical imaging apparatus 1300 includes: A determination unit 1301 for determining a position of the medical accessory in a reference coordinate system based on an image captured by the auxiliary camera; a reconstruction unit 1302 for reconstructing a medical image from the scan results with reference to the reference coordinate system; and a display unit 1303 for displaying the position of the medical accessory in the medical image in real time.

[0087] In some embodiments, the reference coordinate system comprises a world coordinate system, the origin of which is located at the centre of a CT gantry.

[0088] In some embodiments, the medical imaging device further comprises a generation unit 1304 for generating from the scan results a grid matrix on which start points and / or destination points of a planned movement path of the medical accessory are marked, and the display unit 1303 superimposes the grid matrix on the captured image.

[0089] In some embodiments, the display unit 1303 displays information in the grid matrix indicative of the position of the medical accessory and / or the relative position of the medical accessory to the start point and / or the target point in real time.

[0090] In some embodiments, the medical images include three-dimensional images and / or two-dimensional images displayed in one or more planes.

[0091] In some embodiments, the display unit 1303 is responsive to a position of the medical accessory to display the two-dimensional image corresponding to the position of the medical accessory in real time.

[0092] In some embodiments, the two-dimensional image corresponding to the position of the medical accessory comprises a first two-dimensional image passing through the position of the medical accessory and / or a second two-dimensional image perpendicular to the first two-dimensional image passing through the position of the medical accessory and passing through a target point of the medical accessory.

[0093] In some embodiments, the two-dimensional image includes mark information of the target points.

[0094] In some embodiments, the mark information includes information regarding a relative positional relationship between the medical accessory and the target point.

[0095] In some embodiments, the display unit 1303 displays a three-dimensional image of the medical image, superimposed with information indicating a planned movement path of the medical accessory, and / or a relative positional relationship between the medical accessory and a start point and / or a destination point of the movement path, and / or a predicted path determined by the position and extension direction of the medical accessory, and / or the captured image.

[0096] In some embodiments, the determination unit 1301 determines a position of the medical accessory in an imaging coordinate system according to the captured image, and determines a position of the medical accessory in the reference coordinate system according to the position of the medical accessory in the imaging coordinate system and a first transformation matrix between the imaging coordinate system and the reference coordinate system.

[0097] In some embodiments, the medical accessory includes a puncture needle and a marker, and the determination unit 1301 recognizes the marker in the captured image, determines an extension direction and position information of the marker in an imaging coordinate system in which the marker is located, and determines a position of the medical accessory in the imaging coordinate system according to the extension direction and the position information, as well as the relative positional relationship between the marker and the puncture needle.

[0098] In some embodiments, the extension direction of the marker forms a predetermined angle with the extension direction of the puncture needle, and / or the marker is fixed at a predetermined position on the puncture needle, and / or the marker carries information about the medical accessory.

[0099] In some embodiments, the marker is a sleeve fitted around the outer periphery of the needle.

[0100] In some embodiments, the marker comprises at least one of a member having a predetermined temperature, a member having a predetermined shape and / or pattern, or a member that emits light according to a predetermined rule.

[0101] In some embodiments, the reconstruction unit 1302 reconstructs the medical image in the reference coordinate system according to a position of the scan result in the scanning coordinate system and a second transformation matrix between the scanning coordinate system and the reference coordinate system.

[0102] In some embodiments, when the object to be scanned is moved from a first position to a second position for CT scanning, the reconstruction unit 1302 further reconstructs the medical image according to the relative positional relationship between the first position and the second position.

[0103] As can be seen from the above embodiments, the position of the medical accessory in the reference coordinate system is determined based on the captured image obtained by the auxiliary camera, and the medical image is reconstructed from the scanning result based on the reference coordinate system, so that the position of the medical accessory in the medical image can be displayed in real time, which can improve the convenience of operation, help simplify and shorten the positioning process of the medical accessory, and facilitate the operator to quickly and accurately move the medical accessory to the target position.

[0104] An embodiment of the present application further provides a computer device, and FIG. 14 is a schematic diagram of the computer device of the embodiment of the present application. As shown in FIG. 14, the computer device 1400 may include one or more processors (e.g., central processor CPU) 1410 and one or more memories 1420, and the memory 1420 is coupled to the processor 1410. Here, the memory 1420 can store a program 1421 (e.g., a medical imaging program, etc.) and / or data (e.g., captured image, scan result, etc.) and execute the program 1421 under the control of the processor 1410. The memory 1420 may include, for example, a ROM, a floppy disk, a hard disk, an optical disk, a magneto-optical disk, a CD-ROM, or a non-volatile memory card.

[0105] In some embodiments, the functions of the medical imaging device 1300 may be integrated and implemented in the processor 1410. The processor 1410 is configured to implement the medical imaging method described in the above-mentioned embodiments. The embodiment of the processor 1410 can refer to the above-mentioned embodiments, and the description will not be repeated here.

[0106] In some embodiments, the medical imaging device 1300 may be located separately from the processor 1410, for example, the medical imaging device 1300 may be configured as a chip connected to the processor 1410, and the functions of the medical imaging device 1300 may be realized under the control of the processor 1410.

[0107] 14, the computer device 1400 may further include an input device 1430 and a display 1440, among which the functions of the above components are similar to those of the prior art, and the description will not be repeated here. Note that the computer device 1400 does not necessarily need to include all the components shown in FIG. 14, and the computer device 1400 may further include components not shown in FIG. 14, and reference may be made to the related art.

[0108] The processor 1410 may communicate with a medical device, a display, etc. in response to an operation of an input device, and may control the input operation and / or state of the input device. The processor 1410 may be called a microcontroller unit (MCU), a microprocessor, or a microcontroller, or other processor device and / or logic device, and may include a reset circuit, a clock circuit, a chip, a microcontroller, etc. within the processor 1410, and the functions of the processor 1410 may be integrated into the motherboard of the medical device (e.g., the processor 1410 is configured as a chip connected to a motherboard processor (CPU)) or may be provided separately from the motherboard, and the embodiments of the present application are not limited thereto.

[0109] In some embodiments, the computing device may be a computer server or a cloud platform or station, etc., and embodiments of the present application are not limited in this respect.

[0110] The embodiment of the present application further provides a medical imaging system, and FIG. 15 is another schematic diagram of the medical imaging system of the embodiment of the present application, in which the medical imaging system 110 shown in FIG. 15 includes hardware, software, or a combination thereof suitable for supporting medical imaging. For example, the medical imaging system 110 may be a CT system configured to generate and / or render a CT image, or an ultrasound system configured to generate and / or render an ultrasound image, or a magnetic resonance system configured to generate and / or render a magnetic resonance image, etc. As shown in FIG. 15, the medical imaging system 110 may include a processor 113 and a display 114. As described above, the processor 113 may determine the position of the medical accessory in a reference coordinate system based on an image captured by an auxiliary camera, and reconstruct a medical image from a scan result based on the reference coordinate system. The display 114 may display the position of the medical accessory in the medical image in real time. A specific embodiment is as described above, and the description will not be repeated here.

[0111] The display 114 may be configured to display an image (e.g., via a screen). In some cases, the display 114 may also be configured to at least partially generate the image that is displayed. The display 114 may also support user input / output. For example, in addition to an image, the display 114 may also provide user feedback (e.g., via a screen) (e.g., information about the system, its functions, its settings, etc.). The display 114 may also support user input (e.g., via user controls 118) to allow, for example, control of the medical imaging. The user input may relate to controlling the display of images, selecting settings, specifying user preferences, requesting feedback, etc.

[0112] 15, the medical imaging system 110 may further include a scanning device 112, which may be fixed or movable. The scanning device 112 may be configured to generate and / or image a particular type of imaging signal (and / or corresponding data), for example, by moving over an object (or a portion thereof) to be examined, and may include suitable circuitry to perform and / or support such functionality. The scanning device 112 may be an ultrasound probe, an MRI scanner, a CT scanner, or any suitable imaging device.

[0113] In some embodiments, the medical imaging system 110 may further include an imaging device (not shown) capable of acquiring images of objects (e.g., medical accessories and targets) located within its imaging range (e.g., the auxiliary camera described above).

[0114] In some embodiments, the medical imaging system 110 may incorporate additional and dedicated computing resources, such as one or more computing systems 120. In this regard, each computing system 120 may include suitable circuitry, interfaces, logic and / or code for processing, storing and / or communicating data. The computing systems 120 may be dedicated equipment configured specifically for use in connection with medical imaging, or may be general-purpose computing systems (e.g., personal computers, servers, etc.) installed and / or configured to perform the operations described below for the computing systems 120. The computing systems 120 may be configured to support the operation of the medical imaging system 110, as described below. In this regard, various functions and / or operations may be uninstalled from the imaging system. This may simplify and / or centralize certain aspects of the processing to reduce costs (by eliminating the need for increased processing resources in the imaging system).

[0115] The computing systems 120 may be installed and / or arranged for use in different configurations, for example, in some particular embodiments, a single computing system 120 may be used, while in other particular embodiments, multiple computing systems 120 may be configured to operate together (e.g., based on a distributed processing arrangement) or individually, where each computing system 120 is configured to handle particular aspects and / or functions and / or where only a particular medical imaging system 110 is configured to process data.

[0116] In some examples, computing system 120 may be local (e.g., located in conjunction with one or more medical imaging systems 110, e.g., within the same facility and / or same local network), while in other specific implementations computing system 120 may be remote and thus accessible only via a remote connection (e.g., via the Internet or other available remote access technology). In certain specific implementations, computing system 120 may be configured in a cloud-based fashion and may be accessed and / or used in a manner essentially similar to accessing and using other cloud-based systems.

[0117] Once the data is generated and / or configured in the computing system 120, the data can be copied and / or loaded to the medical imaging system 110. For example, the data can be loaded via a directional connection or link between the medical imaging system 110 and the computing system 120. In this regard, communication between the different elements of the configuration can be performed using available wired and / or wireless connections and / or according to any suitable communication (and / or network) standard or protocol. Alternatively or additionally, the data can be indirectly loaded to the medical imaging system 110. For example, the data can be stored on a suitable machine-readable medium (e.g., a flash memory card, etc.) and then the machine-readable medium can be used to load the data to the medical imaging system 110 (in the field, e.g., by a user (e.g., an imaging clinician) or an authorized person of the system), or the data can be downloaded to a locally communicable electronic device (e.g., a laptop, etc.) and then the electronic device can be used in the field (e.g., by a user or authorized person of the system) to upload the data to the medical imaging system 110 via a direct connection (e.g., a USB connector, etc.).

[0118] In operation, the medical imaging system 110 may be used to generate and present (e.g., render or display) images during a medical examination or treatment, and / or in combination with other operations to support user input / output. The images may be 2D, 3D, and / or 4D images. The particular operations or functions performed by the medical imaging system 110 to generate and / or present images will depend on the type of system (i.e., the method for obtaining and / or generating data corresponding to the images).

[0119] Hereinafter, the medical imaging system will be described by way of example with reference to an example in which the medical imaging system is a CT system, and Fig. 16 is a schematic diagram of a CT system 10 according to an embodiment of the present application. As shown in Fig. 16, the system 10 includes a gantry 12, in which an X-ray source 14 and a probe 18 are provided facing each other, and the probe 18 is composed of a plurality of probe modules 20 and a data acquisition system (DAS) 26, and the DAS 26 is used to convert analog data sampled with respect to analog attenuation data received by the plurality of probe modules 20 into a digital signal for use in subsequent processing. The system 10 may further include an imaging device (not shown) for acquiring an image of a medical accessory.

[0120] In some embodiments, the system 10 is used to collect projection data of the object under examination at different angles, and thus the members on the gantry 12 are rotated around a center of rotation 24 to collect the projection data. During the rotation, the X-ray radiation source 14 is used to project X-rays 16 that penetrate the object under examination toward the probe 18, and the attenuated X-ray beam data is pre-processed and then used as projection data of the target volume of the object, so that an image of the object under examination can be reconstructed based on the projection data, and the reconstructed image can display internal features of the object under examination, including, for example, lesions, size, shape, etc. of body tissue structures. The center of rotation 24 of the gantry also defines the center of a scan region 80.

[0121] In some embodiments, the system 10 includes a control mechanism 30. The control mechanism 30 may include an X-ray controller 34 for providing power and timing signals to the X-ray radiation source 14. The control mechanism 30 may further include a gantry controller 32 for controlling a rotational speed and / or a position of the gantry 12 based on an imaging requirement. The control mechanism 30 may further include a bed controller 36 for driving the bed 28 to move to an appropriate position, position the object to be examined on the gantry 12, perform a positioning scan or an axial scan or a bolt scan or other scanning method, etc., and collect projection data of a target volume of the object to be examined. Furthermore, the bed 28 includes a drive device, and the bed controller 36 may control the bed 28 by controlling the drive device.

[0122] The system 10 further includes an image reconstruction module 50, in which the DAS 26 samples and digitizes the projection data collected by the multiple probe modules 20, as described above. The image reconstruction module 50 then performs high-speed image reconstruction based on the sampled and digitized projection data. In some embodiments, the image reconstruction module 50 stores the reconstructed images in the storage device or mass memory 46. Alternatively, the image reconstruction module 50 transmits the reconstructed images to the computer 40 to generate patient information for diagnosis and evaluation. For example, the first positioning image and the diagnostic cross-sectional image are generated based on the projection data collected by the positioning scan, axial scan, bolt scan, or other scanning method.

[0123] 16 is illustrated as a separate entity, in some embodiments, the image reconstruction module 50 may be formed as part of the computer 40. Alternatively, the image reconstruction module 50 may not be present in the system 10, or the computer 40 may perform one or more functions of the image reconstruction module 50. Additionally, the image reconstruction module 50 may be located at a local or remote location and may be connected to the system 10 using a wired or wireless network. In some embodiments, computing resources centralized in a cloud network may be available to the image reconstruction module 50.

[0124] In some embodiments, system 10 further includes a computer 40, and data sampled and digitized by DAS 26 and / or images reconstructed by image reconstruction module 50 are transferred to the computer or computers 40 for processing. In some embodiments, computer 40 stores the data and / or images in a storage device, such as mass memory 46, which may include a hard disk drive, a floppy disk drive, an optical disk read / write (CD-R / W) drive, a digital versatile disk (DVD) drive, a flash drive, and / or a solid state drive. A processor in computer 40 determines a predicted cross-sectional image according to the first registered image.

[0125] In some embodiments, computer 40 transmits the reconstructed image and / or other information to a display 42, which is communicatively coupled to computer 40 and / or image reconstruction module 50. In some embodiments, computer 40 may connect to local or remote displays, printers, workstations and / or similar devices, such as those described above at a medical institution or hospital, or may connect to remote devices via one or more configured wires or a wireless network, such as the Internet and / or a virtual private network. For example, a display may display the predicted cross-sectional image and corresponding scan parameters.

[0126] Computer 40 may also provide commands and parameters to DAS 26 and control mechanisms 30 (including gantry controller 32, x-ray controller 34, and bed controller 36) based on user provided and / or system defined commands and parameters to control system operation, e.g., data collection and / or processing. In some embodiments, computer 40 controls system operation based on user input, e.g., computer 40 may receive user input including commands, scanning protocols, and / or scanning parameters via an operator console 48 connected thereto. Operator console 48 may include a keyboard (not shown) and / or a touch screen that allows a user to enter / select commands, scanning protocols, and / or scanning parameters. In some embodiments, system 10 may include or be connected to a picture archiving and transmission system (PACS) (not shown). In some embodiments, the PACS is further connected to remote systems, such as a radiology information system, a hospital information system, and / or an internal or external network (not shown), to allow an operator at another location to provide commands and parameters and / or access image data.

[0127] The methods or processes of the above-described embodiments may be stored as executable commands in non-volatile memory on a computing device of system 10. For example, computer 40 may include the executable commands in non-volatile memory and apply the medical imaging methods of the present embodiments.

[0128] The computer 40 may be located and / or arranged for use in different configurations. For example, in some implementations, a single computer 40 may be used, while in other implementations, multiple computers 40 may be configured to operate together (e.g., based on a distributed processing configuration) or separately, with each computer 40 configured to handle specific aspects and / or functions and / or process data to generate models used only for a particular system 10. In some implementations, the computer 40 may be local (e.g., in the same location as one or more systems 10, e.g., in the same facility and / or the same local network), while in other implementations, the computer 40 may be remote and thus only accessible via a remote connection (e.g., via the Internet or other available remote access technology). In certain implementations, the computer 40 may be configured in a cloud-like configuration and may be accessed and / or used in a manner essentially similar to how other cloud-based systems are accessed and used.

[0129] Once the data is generated and / or configured, it can be copied and / or loaded into the medical system 10, which can be completed in different ways. For example, the model can be loaded by a directional connection or link between the system 10 and the computer 40. In this regard, the communication between the different elements can be completed using available wired and / or wireless connections and / or according to any suitable communication (and / or network) standard or protocol. Alternatively or additionally, the data can be loaded into the system 10 indirectly. For example, the data can be stored on a suitable machine-readable medium (e.g., a flash memory card, etc.) and then the medium can be used to load the data into the system 10 (in the field, e.g., by a user or an authorized person of the system), or the data can be downloaded to a locally communicable electronic device (e.g., a laptop, etc.) and then the device can be used in the field (e.g., by a user or an authorized person of the system) to upload the data to the system 10 via a direct connection (e.g., a USB connector, etc.).

[0130] An embodiment of the present application further provides a computer readable program, which when executed causes a computer to perform the medical imaging method according to any of the above-mentioned embodiments on the device or system or computing device.

[0131] An embodiment of the present application further provides a storage medium having a computer readable program stored thereon, the computer readable program causing a computer to execute the medical imaging method according to any of the above embodiments in an apparatus or system or computing device.

[0132] The above-mentioned apparatus and method of the present application may be realized in hardware or a combination of hardware and software. The present application relates to such a computer-readable program, which, when executed by a logic element, can cause the logic element to realize the above-mentioned apparatus or component elements, or cause the logic element to realize the above-mentioned various methods or steps. The present application also relates to a storage medium for storing the above-mentioned programs, such as a hard disk, a magnetic disk, an optical disk, a DVD, a flash memory, etc.

[0133] The methods / apparatuses described in relation to the embodiments of the present application may be implemented as hardware, software modules executed by a processor, or a combination of both. For example, one or more of the illustrated functional blocks and / or one or more combinations of the functional blocks may correspond to each software module of a computer program flow, or each hardware module. These software modules may correspond to each illustrated step. These hardware modules can be implemented by writing these software modules using, for example, a field programmable gate array (FPGA).

[0134] The software module may be located in RAM memory, flash memory, ROM memory, EPROM memory, EEPROM memory, registers, hard disk, removable disk, CD-ROM, or any other form of storage medium known in the art. The storage medium may be coupled to the processor so that the processor can read information from and write information to the storage medium, or the storage medium may be a component of the processor. The processor and the storage medium may be located in an ASIC. The software module may be stored in the memory of the mobile terminal, or in a memory card that can be inserted into the mobile terminal. For example, if the device (e.g., the mobile terminal) employs a large-capacity MEGA-SIM card or a large-capacity flash memory device, the software module may be stored in the MEGA-SIM card or the large-capacity flash memory device.

[0135] One or more of the functional blocks and / or one or more combinations of the functional blocks illustrated in the drawings may be implemented as a general purpose processor, a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic device, a discrete gate or transistor logic device, a discrete hardware assembly, or any suitable combination thereof to perform the functions described herein. One or more of the functional blocks and / or one or more combinations of the functional blocks illustrated in the drawings may also be implemented as a general purpose processor, a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic device, a discrete gate or transistor logic device, a discrete hardware assembly, or any suitable combination thereof to perform the functions described herein. The present invention may be realized as a combination of computing devices, such as a combination of a DSP and a microprocessor, multiple microprocessors, one or more microprocessors in communication with a DSP, or any other such configuration. The above embodiments are merely illustrative of the embodiments of the present application, but the present application is not limited thereto and may be modified as appropriate based on the above embodiments. For example, the above embodiments may be used alone, or one or more of the above embodiments may be combined.

[0136] Although the present application has been described in relation to specific embodiments, it is obvious to those skilled in the art that these descriptions are merely illustrative and do not limit the scope of protection of the present application. Those skilled in the art can make various modifications and alterations to the present application based on the spirit and principles of the present application, and these modifications and alterations are also within the scope of the present application.

[0137] The preferred embodiments of the present application have been described above with reference to the drawings. Since many features and advantages of these embodiments are apparent from this detailed description, the appended claims are intended to cover all such features and advantages within the true spirit and scope of these embodiments. In addition, since many modifications and variations can be easily conceived by those skilled in the art, the present application is not limited to the exact structure and operation illustrated and described, but can cover all appropriate modifications, variations and equivalents within the scope thereof.

Claims

1. 1. A medical imaging method comprising: determining a position of the medical accessory in a reference coordinate system based on the captured image acquired by the auxiliary camera; reconstructing a medical image from the scan results with reference to the reference coordinate system; and displaying in real time a position of the medical accessory in the medical image.

2. The method of claim 1 , wherein the reference coordinate system comprises a world coordinate system, the origin of the world coordinate system being located at the center of a scan gantry.

3. The method comprises: generating a grid matrix from the scan results, the grid matrix being marked with start and / or end points of a planned path of movement of the medical accessory; 2. The method of claim 1, further comprising: overlaying the grid matrix on the captured image.

4. The method comprises:

4. The method of claim 3, further comprising displaying in the grid matrix information indicating a position of the medical accessory and / or a relative positional relationship between the medical accessory and the start point and / or the target point in real time.

5. The method of claim 1 , wherein the medical images include three-dimensional images and / or two-dimensional images displayed in one or more planes.

6. The method comprises:

6. The method of claim 5, further comprising displaying the two-dimensional image corresponding to the position of the medical accessory in real time in response to a position of the medical accessory.

7. The two-dimensional image corresponding to the position of the medical accessory is 7. The method of claim 6, further comprising: a first two-dimensional image passing through the position of the medical accessory and / or a second two-dimensional image perpendicular to the first two-dimensional image passing through the position of the medical accessory and passing through a target point of the medical accessory.

8. The method of claim 7 , wherein the second two-dimensional image includes mark information of the target point.

9. The method of claim 8 , wherein the mark information includes information regarding a relative positional relationship between the medical accessory and the target point.

10. The method comprises: The method according to claim 5, further comprising superimposing and displaying, on the three-dimensional image of the medical image, a planned movement path of the medical accessory, and / or information indicating a relative positional relationship between the medical accessory and a start point and / or a target point of the movement path, and / or a predicted path determined by the position and extension direction of the medical accessory, and / or the captured image.

11. Determining a position of the medical accessory in a reference coordinate system based on the captured image acquired by the auxiliary camera determining a position of the medical accessory in an imaging coordinate system from the captured image; and determining a position of the medical accessory in the reference coordinate system in response to a position of the medical accessory in the imaging coordinate system and a first transformation matrix between the imaging coordinate system and the reference coordinate system.

12. The medical accessory includes a puncture needle and a marker, Determining a position of the medical accessory in an imaging coordinate system from the captured image includes: Recognizing the marker in the captured image and determining an extension direction and position information of the marker in the image capture coordinate system; The method according to claim 11, further comprising determining a position of the puncture needle in the imaging coordinate system according to the extension direction, the position information, and a relative positional relationship between the marker and the puncture needle.

13. The method according to claim 12, characterized in that the extension direction of the marker and the extension direction of the puncture needle form a predetermined angle, and / or the marker is fixed at a predetermined position on the puncture needle, and / or the marker carries information regarding the medical accessory.

14. The method according to claim 13, wherein the marker is a sleeve fitted around the outer periphery of the puncture needle.

15. The marker is 13. The method of claim 12, comprising at least one of a member having a predetermined temperature, a member having a predetermined shape and / or pattern, or a member that emits light according to a predetermined rule.

16. Reconstructing a medical image from the scan results with reference to the reference coordinate system includes:

2. The method of claim 1, further comprising reconstructing the medical image in the reference coordinate system according to a position of the scan result in the scanning coordinate system and a second transformation matrix between the scanning coordinate system and the reference coordinate system.

17. 17. The method of claim 16, further comprising the step of: when a scanned object is moved from a first position to a second position, reconstructing the medical image according to a relative positional relationship between the first position and the second position.

18. 1. A medical imaging device comprising: a determination unit for determining a position of the medical accessory in a reference coordinate system based on the captured images acquired by the auxiliary camera; a reconstruction unit for reconstructing a medical image from a scan result based on the reference coordinate system; and a display unit for displaying in real time the location of said medical accessory in said medical image.

19. The apparatus comprises: a generating unit for generating from said scanning results a grid matrix in which start and / or end points of a planned movement path of said medical accessory are marked, The apparatus of claim 18 , wherein the display unit overlays the grid matrix on the captured image.

20. 20. The apparatus of claim 18, wherein the medical images include three-dimensional images and / or two-dimensional images displayed in one or more planes.

21. 21. The apparatus of claim 20, wherein the display unit is operable to display the two-dimensional image corresponding to the position of the medical accessory in real time in response to a position of the medical accessory.

22. The device according to claim 20, characterized in that the display unit displays, on a three-dimensional image of the medical image, a planned movement path of the medical accessory, and / or information indicating a relative positional relationship between the medical accessory and a start point and / or a target point of the movement path, and / or a predicted path determined by the position and extension direction of the medical accessory, and / or the captured image, superimposed thereon.

23. The device according to claim 18, characterized in that the medical accessory includes a puncture needle and a marker, the extension direction of the marker and the extension direction of the puncture needle form a predetermined angle, and / or the marker is fixed to a predetermined position of the puncture needle, and / or the marker carries information regarding the medical accessory.

24. 24. The device according to claim 23, wherein the marker is a sleeve fitted onto the outer periphery of the puncture needle.

25. 1. A medical imaging system comprising: a processor for determining a position of the medical accessory in a reference coordinate system based on the captured images obtained by the auxiliary camera, and for reconstructing a medical image from the scan results with reference to the reference coordinate system; and a display that displays in real time the location of the medical accessory in the medical image.

Images