A surgical path imaging display method, device and related equipment

Abstract

The present application relates to the field of medical imaging technology, and in particular to an imaging display method, device and related equipment for a surgical path. The method includes: obtaining a CT image and a mixed reality image containing a positioning mark; the spatial position of the positioning mark when imaged is fixed; determining the first stereo coordinate set and the second stereo coordinate set of the positioning mark respectively according to the CT image and the mixed reality image containing the positioning mark; determining a coordinate conversion matrix according to the first stereo coordinate set and the second stereo coordinate set of the positioning mark; converting the first stereo coordinate set of the surgical path into the second stereo coordinate set of the surgical path through the coordinate conversion matrix; and performing imaging display according to the second stereo coordinate set of the surgical path. The present application can solve the problem that the existing imaging display method of the surgical path is cumbersome to operate.

Description

Technical Field

[0001] The present application relates to the field of medical imaging technology, and in particular to an imaging display method, device and related equipment for a surgical path. Background Art

[0002] In the prior art, in order to improve the success rate of surgery, a stereoscopic image of the surgical path and a stereoscopic image of the lesion can be displayed to the doctor at the surgical site to indicate the relative position between the stereoscopic image of the surgical path and the stereoscopic image of the lesion for the doctor's reference. The stereoscopic image of the surgical path is a linear pattern drawn in advance at the stereoscopic image of the lesion displayed by the application software, which can indicate the entry point and direction of the scalpel in the actual lesion area of ​​the patient.

[0003] In actual applications, the application software sends the stereo coordinate sets of the surgical path stereo image and the lesion stereo image to the flat display device, and the flat display device displays the surgical path stereo image and the lesion stereo image according to the stereo coordinate set. In actual operation, even if the flat display device can display the surgical path stereo image and the lesion stereo image in high definition, it is known from experience that it is difficult for the flat display device to fully and clearly display the relative position relationship between the two stereo images. Sometimes, the display angles of the two stereo images need to be adjusted multiple times to determine the relative position between the two stereo images. In addition, since doctors usually refer to the surgical path stereo image during the operation, the behavior of always paying attention to the flat display device may also distract the doctor. Since the operation time is precious, it is known that the existing surgical path imaging display method has the problem of cumbersome operation. Summary of the invention

[0004] In view of this, the purpose of the present application is to provide a surgical path imaging display method, device and related equipment to solve the problem that the existing surgical path imaging display method has troublesome operation.

[0005] In a first aspect, the present application provides a method for imaging and displaying a surgical path, the method comprising:

[0006] Acquire a CT image and a mixed reality image containing a positioning mark; the spatial position of the positioning mark when imaged is fixed;

[0007] Determine a first stereoscopic coordinate set and a second stereoscopic coordinate set of the positioning mark respectively according to the CT image and the mixed reality image containing the positioning mark;

[0008] Determine a coordinate conversion matrix according to the first three-dimensional coordinate set and the second three-dimensional coordinate set of the positioning mark;

[0009] The first stereoscopic coordinate set of the surgical path is converted into a second stereoscopic coordinate set of the surgical path by using the coordinate conversion matrix; the first stereoscopic coordinate set of the surgical path is determined according to a stereoscopic image of the surgical path, and the stereoscopic image of the surgical path is determined according to a CT image containing a lesion area;

[0010] Imaging display is performed according to the second stereoscopic coordinate set of the surgical path.

[0011] Preferably, the positioning mark includes: a first positioning mark and a second positioning mark; the relative position between the first positioning mark and the second positioning mark is fixed; the first positioning mark is used for identification and imaging by a CT machine, and the second positioning mark is used for identification and imaging by a mixed reality head-mounted device;

[0012] The step of acquiring a CT image and a mixed reality image containing positioning marks includes:

[0013] The positioning mark is imaged by a CT machine and a mixed reality head-mounted device respectively to obtain a CT image containing the first positioning mark and a mixed reality image containing the second positioning mark.

[0014] Preferably, each of the positioning marks includes the same number of first positioning marks and second positioning marks, each of the first positioning marks is used in conjunction with one of the second positioning marks, and the relative positions of the first positioning marks and the second positioning marks used in conjunction are fixed.

[0015] Preferably, determining the first stereoscopic coordinate set and the second stereoscopic coordinate set of the positioning mark respectively according to the CT image and the mixed reality image containing the positioning mark comprises:

[0016] Determine a first stereoscopic coordinate set of a plurality of first positioning marks according to a CT image including a plurality of first positioning marks; determine a second stereoscopic coordinate set of a plurality of second positioning marks according to a mixed reality image including a plurality of second positioning marks;

[0017] The step of determining a coordinate transformation matrix according to the first three-dimensional coordinate set and the second three-dimensional coordinate set of the positioning mark comprises:

[0018] Determine the first three-dimensional coordinate set of the plurality of second positioning marks according to the first three-dimensional coordinate set of the plurality of first positioning marks and the relative position data between the plurality of first positioning marks and the second positioning marks used in conjunction with them; the relative position data indicates the relative position between the first positioning mark and the second positioning mark used in conjunction with them;

[0019] Based on the first stereoscopic coordinate set and the second stereoscopic coordinate set of the plurality of second positioning marks, a coordinate conversion matrix between the first stereoscopic coordinate system based on which the CT image is imaged and the second stereoscopic coordinate system based on which the mixed reality image is imaged is determined.

[0020] In a second aspect, the present application provides an imaging display device for a surgical path, the device comprising: an image module, a coordinate module, a matrix module, a conversion module and an imaging module;

[0021] The image module is used to obtain a CT image and a mixed reality image containing a positioning mark; the spatial position of the positioning mark when imaged is fixed;

[0022] The coordinate module is used to determine the first stereoscopic coordinate set and the second stereoscopic coordinate set of the positioning mark respectively according to the CT image and the mixed reality image containing the positioning mark;

[0023] The matrix module is used to determine a coordinate transformation matrix according to the first three-dimensional coordinate set and the second three-dimensional coordinate set of the positioning mark;

[0024] The conversion module is used to convert a first stereoscopic coordinate set of the surgical path into a second stereoscopic coordinate set of the surgical path through the coordinate conversion matrix; the first stereoscopic coordinate set of the surgical path is determined according to a stereoscopic image of the surgical path, and the stereoscopic image of the surgical path is determined according to a CT image containing a lesion area;

[0025] The imaging module is used for performing imaging display according to the second stereoscopic coordinate set of the surgical path.

[0026] Preferably, the positioning mark includes: a first positioning mark and a second positioning mark; the relative position between the first positioning mark and the second positioning mark is fixed; the first positioning mark is used for identification and imaging by a CT machine, and the second positioning mark is used for identification and imaging by a mixed reality head-mounted device;

[0027] The image module is also used to image the positioning mark through a CT machine and a mixed reality head-mounted device to obtain a CT image containing the first positioning mark and a mixed reality image containing the second positioning mark.

[0028] Preferably, each of the positioning marks includes the same number of first positioning marks and second positioning marks, each of the first positioning marks is used in conjunction with one of the second positioning marks, and the relative positions of the first positioning marks and the second positioning marks used in conjunction are fixed.

[0029] In a third aspect, the present application provides an electronic device, comprising a processor and a memory, wherein the memory is used to store software programs, and the processor runs or executes the software programs stored in the memory so that the electronic device implements the above-mentioned method.

[0030] In a fourth aspect, the present application provides a computer-readable storage medium, wherein the computer-readable storage medium is used to store program codes executed by a processor, wherein the program codes include methods for implementing the above-mentioned method.

[0031] In a fifth aspect, the present application provides a computer program product, including a program code, so that when a computer runs the computer program product, the computer implements the above method.

[0032] Beneficial effects:

[0033] The present application provides an imaging display method, device and related equipment for a surgical path, the method comprising: obtaining a CT image and a mixed reality image containing a positioning mark; determining the first stereo coordinate set and the second stereo coordinate set of the positioning mark respectively according to the CT image and the mixed reality image containing the positioning mark; determining a coordinate conversion matrix according to the first stereo coordinate set and the second stereo coordinate set of the positioning mark; converting the first stereo coordinate set of the surgical path into the second stereo coordinate set of the surgical path through the coordinate conversion matrix; the first stereo coordinate set of the surgical path is determined according to the stereo image of the surgical path, and the stereo image of the surgical path is determined according to the CT image containing the lesion area; imaging and displaying according to the second stereo coordinate set of the surgical path. In summary, the present application determines the coordinate conversion matrix through the CT image and the mixed reality image containing the positioning mark, and then converts the first stereo coordinate set of the surgical path drawn in advance into the second stereo coordinate set through the coordinate conversion matrix, and then uses the mixed reality device to perform imaging and display according to the second stereo coordinate set of the surgical path, so as to obtain a mixed reality image of the surgical path, and the doctor can subsequently perform surgery according to the mixed reality image of the surgical path displayed by the mixed reality head-mounted device. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments of the present application will be briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0035] Figure 1 A schematic diagram of the connection between the mixed reality head mounted device, application software and CT machine provided in the embodiment of the present application;

[0036] Figure 2 A schematic diagram of a flow chart of a method for imaging and displaying a surgical path provided in an embodiment of the present application;

[0037] Figure 3 A schematic diagram of the structure of a positioning mark provided in an embodiment of the present application;

[0038] Figure 4 A schematic diagram of the structure of an imaging display device for a surgical path provided in an embodiment of the present application. DETAILED DESCRIPTION

[0039] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution of the present application will be clearly and completely described below in conjunction with the accompanying drawings. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present application.

[0040] The imaging display method of the surgical path in the embodiment of the present application is implemented based on the imaging display system of the surgical path. The imaging display system of the surgical path includes: a mixed reality head-mounted device, application software (installed on a server) and a CT machine, wherein the application software is respectively connected to the CT machine and the mixed reality head-mounted device for communication. Figure 1 As shown, Figure 1 A schematic diagram of the connection between the mixed reality head mounted device, the application software and the CT machine provided in the embodiment of the present application, wherein the CT machine takes multiple CT images containing the lesion area and transmits the multiple CT images containing the lesion area to the application software.

[0041] First, the present application provides a first embodiment of a method for imaging and displaying a surgical path. The essence of the technical solution of the embodiment of the present application is to render the surgical path determined by the application software through a mixed reality wearable device to obtain a mixed reality image of the surgical path, so that the mixed reality avatar guides the doctor to perform the surgical operation, such as Figure 2 As shown, Figure 2 A schematic diagram of a flow chart of a method for imaging and displaying a surgical path provided in an embodiment of the present application, the method comprising:

[0042] S110: Acquire a CT image and a mixed reality image containing positioning marks.

[0043] The spatial position of the positioning mark when it is imaged is fixed.

[0044] Specifically, the positioning mark is a movable physical structure with a fixed shape. The CT image containing the positioning mark is obtained by photographing the positioning mark with a CT machine, and the mixed reality image containing the positioning mark is obtained by photographing the positioning mark with a mixed reality head-mounted device.

[0045] In actual operation, the positioning mark needs to be placed in the target operating room, which is the operating room where the doctor performs the operation related to the surgical path. The positioning mark is placed next to the patient to be scanned by CT in the operating room, and can also be placed next to the CT bed.

[0046] It should be emphasized that when acquiring CT images and mixed reality images including positioning marks, the placement position and placement direction of the positioning marks in the target operating room are fixed, that is, the spatial position of the positioning mark when it is imaged is fixed, which is used to ensure that the mechanical coordinates of the positioning mark in the CT image and mixed reality image containing the positioning mark are the same.

[0047] S120: Determine a first stereoscopic coordinate set and a second stereoscopic coordinate set of the positioning mark respectively according to the CT image and the mixed reality image containing the positioning mark.

[0048] Specifically, the first stereo coordinate set of the positioning mark is determined based on the first stereo coordinate system, and the second stereo coordinate set of the positioning mark is determined based on the second stereo coordinate system. The first stereo coordinate system is a coordinate system constructed by the application software based on the CT photos taken by the CT machine set in the target operating room, and the second stereo coordinate system is a coordinate system constructed by the mixed reality head-mounted device based on the target operating room where the CT machine is located. In actual applications, the first stereo coordinate system can be adjusted and updated according to the settings of the CT machine, and the second stereo coordinate system is updated according to different environments (such as different operating rooms). In the embodiment of the present application, it is assumed that the technical solution of the present application is implemented based on the constructed first stereo coordinate system and second stereo coordinate system.

[0049] In actual operation, when there is a need to display a mixed reality image of a surgical path through a mixed reality head mounted device, it is first necessary to draw a stereoscopic image of the surgical path on a stereoscopic image of the patient's lesion through application software, and the stereoscopic image of the lesion is modeled based on multiple CT images including the patient's lesion area. After the stereoscopic image of the surgical path is drawn, the application software determines a first stereoscopic coordinate set of the surgical path based on the stereoscopic image of the surgical path and sends the first stereoscopic coordinate set of the surgical path to the mixed reality head mounted device.

[0050] In actual operation, the mixed reality head mounted device captures a mixed reality image containing a positioning mark, and then determines a second stereo coordinate set of the positioning mark based on the mixed reality image containing the positioning mark. It should be emphasized that there is no requirement for the order of determining the first stereo coordinate set and the second stereo coordinate set of the positioning mark, and the execution order can be determined according to actual needs, and this application does not make specific limitations.

[0051] S130: Determine a coordinate conversion matrix according to the first three-dimensional coordinate set and the second three-dimensional coordinate set of the positioning mark.

[0052] Specifically, after the first three-dimensional coordinate set and the second three-dimensional coordinate set of the positioning mark are determined, the coordinate conversion matrix between the first three-dimensional coordinate system and the second three-dimensional coordinate system can be determined.

[0053] In actual operation, if the first stereo coordinate set of the surgical path in the first stereo coordinate system is not converted into its first stereo coordinate set in the second stereo coordinate system through the coordinate conversion matrix, if the mixed reality head-mounted device directly renders the image according to the first stereo coordinate set of the surgical path transmitted by the application software, it is impossible to ensure that the mixed reality avatar of the surgical path obtained by imaging (the position in the target operating room) is completely consistent with the stereoscopic image of the surgical path drawn at the stereoscopic image of the lesion through the application software (the position in the target operating room). Therefore, in actual operation, it is necessary to convert the first stereo coordinate set of the surgical path into the second stereo coordinate set through the coordinate conversion matrix.

[0054] S140: Convert the first three-dimensional coordinate set of the surgical path into a second three-dimensional coordinate set of the surgical path through a coordinate conversion matrix.

[0055] The first stereoscopic coordinate set of the surgical path is determined based on a stereoscopic image of the surgical path, and the stereoscopic image of the surgical path is determined based on a CT image containing a lesion area.

[0056] Specifically, after receiving the first stereoscopic coordinate set of the surgical path, the mixed reality head mounted device converts the first stereoscopic coordinate set of the surgical path into a second stereoscopic coordinate set of the surgical path through a coordinate conversion matrix.

[0057] S150: Perform imaging display according to the second stereoscopic coordinate set of the surgical path.

[0058] Specifically, the mixed reality head mounted device renders the surgical path according to the second stereoscopic coordinate set of the surgical path to obtain a mixed reality image of the surgical path, and guides the doctor to perform the surgical operation based on the mixed reality image of the surgical path. In one implementation, the surgical operation may be a puncture operation, and the surgical path may be a puncture path.

[0059] In one implementation, after obtaining the mixed reality image of the surgical path, it is necessary to determine whether the position between the mixed reality image of the surgical path and the mixed reality image of the lesion area displayed by the mixed reality head-mounted device, and the position between the stereoscopic image of the surgical path displayed in the application software and the stereoscopic image of the lesion are consistent. If not, step S110 needs to be re-executed.

[0060] In summary, the embodiment of the present application converts the first stereoscopic coordinate set of the surgical path determined by the application software into a second stereoscopic coordinate set through the mixed reality head-mounted device. The mixed reality head-mounted device performs imaging according to the second stereoscopic coordinate set of the surgical path to obtain a mixed reality image of the surgical path, so that the doctor wearing the mixed reality head-mounted device can be stably guided by the mixed reality image of the surgical path without having to pay attention to a separately provided flat-panel display device at all times. Therefore, the surgical efficiency and quality can be significantly improved.

[0061] Second, the present application provides a second embodiment of the imaging display method of the surgical path. In the embodiment of the present application, each positioning mark includes an equal number of first positioning marks and second positioning marks, each first positioning mark is used in conjunction with a second positioning mark, and the relative positions between the first positioning marks and the second positioning marks used in conjunction are fixed.

[0062] And, in the second embodiment of the present application, S110 includes step (1), the details of which are as follows:

[0063] Step (1): The positioning mark is imaged by a CT machine and a mixed reality head mounted device respectively to obtain a CT image containing a first positioning mark and a mixed reality image containing a second positioning mark.

[0064] Specifically, in practical applications, the number of the first positioning mark and the second positioning mark is at least three, which is used to ensure the accuracy of the coordinate transformation matrix determined based on the first positioning mark and the second positioning mark. In an embodiment of the present application, the first positioning mark and the second positioning mark can both be set to four. The relative position between the first positioning mark and the second positioning mark is fixed.

[0065] In actual operation, Figure 3 As shown, Figure 3 A schematic diagram of the structure of the positioning mark provided in the embodiment of the present application, wherein the first positioning mark is a metal ring that can be recognized and imaged by a CT machine, and the second positioning mark is a reflective film that can be recognized and imaged by a mixed reality head-mounted device ( Figure 3 The reflective film is attached to the center of the metal ring.

[0066] The multiple metal rings are connected by connecting rods. The purpose of the connection is to keep the relative positions of the multiple metal rings relatively fixed. The material of the connecting rod can be wood or other plastic materials. It should be emphasized here that the metal rings and reflective films are the first positioning marks and second positioning marks exemplarily exhibited in the embodiments of this application. In actual operation, they can be equivalently replaced by other objects that can be recognized by CT machines and mixed reality head-mounted devices.

[0067] And, in the second embodiment of the present application, S120 includes step (2), the details of which are as follows:

[0068] Step (2): Determine a first stereoscopic coordinate set of a plurality of first positioning marks based on a CT image containing a plurality of first positioning marks. Determine a second stereoscopic coordinate set of a plurality of second positioning marks based on a mixed reality image containing a plurality of second positioning marks.

[0069] Specifically, the CT machine captures multiple CT images containing multiple first positioning marks, and the CT machine transmits the multiple CT images containing multiple first positioning marks to the application software, and the application software determines the first three-dimensional coordinate set of the multiple first positioning marks based on the multiple CT images containing the multiple first positioning marks.

[0070] The mixed reality head mounted device captures a plurality of mixed reality images including a plurality of second positioning marks and determines a second stereoscopic coordinate set of the plurality of second positioning marks based on the plurality of mixed reality images including the plurality of second positioning marks.

[0071] In actual operation, after a plurality of CT images of the four metal rings are obtained by a CT machine, application software performs modeling according to the plurality of CT images of the four metal rings, and accordingly obtains a first stereoscopic coordinate set of the four metal rings.

[0072] The mixed reality head-mounted device captures multiple mixed reality images including four reflective films and determines a set of second three-dimensional coordinate sets of the reflective films {p′1, p′2, p′3, p′4} based on the multiple mixed reality images of the four reflective films, where {p′1, p′2, p′3, p′4} are the second three-dimensional coordinates of multiple second positioning marks; wherein p′1, p′2, p′3, and p′4 respectively represent the second three-dimensional coordinate sets of different reflective films.

[0073] And, in the second embodiment of the present application, S130 includes steps (3) to (4), the details of which are as follows:

[0074] Step (3): Determine the first three-dimensional coordinate set of the plurality of second positioning marks based on the first three-dimensional coordinate set of the plurality of first positioning marks and the relative position data between the plurality of first positioning marks and the second positioning marks used in conjunction therewith.

[0075] The relative position data indicates the relative position between the first positioning mark and the second positioning mark used in conjunction therewith.

[0076] Specifically, the application software determines the first stereoscopic coordinate set of the second positioning mark according to the first stereoscopic coordinate set of the first positioning mark and the relative position data between the first positioning mark and the second positioning mark; the application software then transmits the first stereoscopic coordinate set of the second positioning mark to the mixed reality head mounted device. In actual operation, the relative position data can be the distance between the first positioning mark and the second positioning mark or other data that can clearly indicate the relative position between the first positioning mark and the second positioning mark.

[0077] In actual operation, the application software determines the set of first 3D coordinate sets of the reflective film located in the center of the four metal rings {p1, p2, p3, p4} based on the first 3D coordinate sets of the four metal rings and the relative position data between the metal rings and the reflective film. {p1, p2, p3, p4} is the first 3D coordinate set of multiple first positioning marks; wherein p1, p2, p3, p4 represent the first 3D coordinate sets of different reflective films respectively. The application software sends the set of first 3D coordinate sets of the four reflective films to the mixed reality head mounted device.

[0078] Step (4): Based on the first stereoscopic coordinate set and the second stereoscopic coordinate set of multiple second positioning marks, determine the coordinate transformation matrix between the first stereoscopic coordinate system based on which the CT image is imaged and the second stereoscopic coordinate system based on which the mixed reality image is imaged.

[0079] Specifically, in actual operation, the mixed reality device determines the coordinate transformation matrix according to the first stereo coordinate set {p1, p2, p3, p4} and the second stereo coordinate set {p′1, p′2, p′3, p′4} of the reflective film.

[0080] Third, the present application provides an embodiment of an imaging display device for a surgical path, which is applied to a mixed reality head-mounted device, the mixed reality head-mounted device is connected to an application software, and the application software is also connected to a CT machine, such as Figure 4 As shown, Figure 4 This is a schematic diagram of the structure of an imaging display device for a surgical path provided in an embodiment of the present application. The device includes: an image module 310, a coordinate module 320, a matrix module 330, a conversion module 340 and an imaging module 350.

[0081] The image module 310 is used to obtain a CT image or a mixed reality image containing a positioning mark; the spatial position of the positioning mark when it is imaged is fixed.

[0082] The coordinate module 320 is used to determine the first stereoscopic coordinate set and the second stereoscopic coordinate set of the positioning mark according to the CT image and the mixed reality image containing the positioning mark.

[0083] The matrix module 330 is used to determine a coordinate transformation matrix according to the first three-dimensional coordinate set and the second three-dimensional coordinate set of the positioning mark.

[0084] The conversion module 340 is used to convert the first stereoscopic coordinate set of the surgical path into the second stereoscopic coordinate set of the surgical path through a coordinate conversion matrix; the first stereoscopic coordinate set of the surgical path is determined based on the stereoscopic image of the surgical path, and the stereoscopic image of the surgical path is determined based on the CT image containing the lesion area.

[0085] The imaging module 350 is used to perform imaging display according to the second stereoscopic coordinate set of the surgical path.

[0086] In one implementation, the positioning mark includes: a first positioning mark and a second positioning mark; the relative position between the first positioning mark and the second positioning mark is fixed; the first positioning mark is used for identification and imaging by a CT machine, and the second positioning mark is used for identification and imaging by a mixed reality head-mounted device.

[0087] The image module 310 is also used to image the positioning mark through a CT machine and a mixed reality head-mounted device to obtain a CT image containing the first positioning mark and a mixed reality image containing the second positioning mark.

[0088] In one implementation, each positioning mark includes the same number of first positioning marks and second positioning marks, each first positioning mark is used in conjunction with a second positioning mark, and the relative positions of the first positioning marks and the second positioning marks used in conjunction are fixed.

[0089] In one implementation, the coordinate module 320 is further configured to determine a first stereoscopic coordinate set of a plurality of first positioning marks according to a CT image including a plurality of first positioning marks; and determine a second stereoscopic coordinate set of a plurality of second positioning marks according to a mixed reality image including a plurality of second positioning marks;

[0090] The coordinate module 320 is further used to determine the first three-dimensional coordinate set of the plurality of second positioning marks according to the first three-dimensional coordinate set of the plurality of first positioning marks and the relative position data between the plurality of first positioning marks and the second positioning marks used in conjunction therewith;

[0091] The matrix module 330 is also used to determine the coordinate transformation matrix between the first stereoscopic coordinate system based on which the CT image is imaged and the second stereoscopic coordinate system based on which the mixed reality image is imaged based on the first stereoscopic coordinate set and the second stereoscopic coordinate set of multiple second positioning marks.

[0092] Fourth, the present application also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements steps S110 to S150 provided in the above embodiment when executing the computer program.

[0093] Fifth, the present application also provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, steps S110 to S150 of the above embodiment are executed.

[0094] Sixth, the computer program product provided in the present application includes a computer-readable storage medium storing program code. The instructions included in the program code can be used to execute the method in the previous method embodiment. The specific implementation can refer to steps S110 to S150 of the method embodiment, which will not be repeated here.

[0095] In the embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are merely schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some communication interfaces, and the indirect coupling or communication connection of the devices or units can be electrical, mechanical or other forms.

[0096] In addition, the units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0097] Furthermore, the functional modules in the various embodiments of the present application may be integrated together to form an independent part, or each module may exist separately, or two or more modules may be integrated to form an independent part.

[0098] It should be noted that if the function is implemented in the form of a software function module and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application can essentially be embodied in the form of a software product, or the part that contributes to the prior art or the part of the technical solution. The computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM) random access memory (RAM), disk or optical disk, and other media that can store program codes.

[0099] In this document, relational terms such as first and second, etc. are used merely to distinguish one entity or operation from another entity or operation, but do not necessarily require or imply any such actual relationship or order between these entities or operations.

[0100] The above description is only an embodiment of the present application and is not intended to limit the protection scope of the present application. For those skilled in the art, the present application may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A method for imaging and displaying a surgical path, characterized in that: The method comprises: Acquire a CT image containing a positioning mark and a mixed reality image containing a positioning mark; the spatial position of the positioning mark when imaged is fixed; Determine a first stereoscopic coordinate set and a second stereoscopic coordinate set of the positioning mark respectively according to the CT image containing the positioning mark and the mixed reality image containing the positioning mark; Determine a coordinate conversion matrix according to the first three-dimensional coordinate set and the second three-dimensional coordinate set of the positioning mark; The first stereoscopic coordinate set of the surgical path is converted into a second stereoscopic coordinate set of the surgical path by using the coordinate conversion matrix; the first stereoscopic coordinate set of the surgical path is determined according to a stereoscopic image of the surgical path, and the stereoscopic image of the surgical path is determined according to a CT image containing a lesion area; Perform imaging display according to a second stereoscopic coordinate set of the surgical path; The positioning mark includes: a first positioning mark and a second positioning mark; the relative position between the first positioning mark and the second positioning mark is fixed; the first positioning mark is used for identification and imaging by a CT machine, and the second positioning mark is used for identification and imaging by a mixed reality head-mounted device; The step of determining a coordinate transformation matrix according to the first three-dimensional coordinate set and the second three-dimensional coordinate set of the positioning mark comprises: Determine the first three-dimensional coordinate set of the plurality of second positioning marks according to the first three-dimensional coordinate set of the plurality of first positioning marks and the relative position data between the plurality of first positioning marks and the second positioning marks used in conjunction with them; the relative position data indicates the relative position between the first positioning mark and the second positioning mark used in conjunction with them; Based on the first stereoscopic coordinate set and the second stereoscopic coordinate set of the plurality of second positioning marks, a coordinate conversion matrix between the first stereoscopic coordinate system based on which the CT image is imaged and the second stereoscopic coordinate system based on which the mixed reality image is imaged is determined.

2. The method according to claim 1, characterized in that The acquiring of the CT image containing the positioning mark and the mixed reality image containing the positioning mark comprises: The positioning mark is imaged by a CT machine and a mixed reality head-mounted device respectively to obtain a CT image containing the first positioning mark and a mixed reality image containing the second positioning mark.

3. The method according to claim 2, characterized in that Each of the positioning marks includes the same number of first positioning marks and second positioning marks, each of the first positioning marks is used in conjunction with one of the second positioning marks, and the relative positions of the first positioning marks and the second positioning marks used in conjunction are fixed.

4. The method according to claim 3, characterized in that The method of determining the first stereoscopic coordinate set and the second stereoscopic coordinate set of the positioning mark respectively according to the CT image containing the positioning mark and the mixed reality image containing the positioning mark comprises: A first stereoscopic coordinate set of multiple first positioning marks is determined based on a CT image containing multiple first positioning marks; and a second stereoscopic coordinate set of multiple second positioning marks is determined based on a mixed reality image containing multiple second positioning marks.

5. An imaging display device for a surgical path, characterized in that: The device comprises: an image module, a coordinate module, a matrix module, a conversion module and an imaging module; The image module is used to obtain a CT image containing a positioning mark and a mixed reality image containing a positioning mark; the spatial position of the positioning mark when it is imaged is fixed; The coordinate module is used to determine a first stereoscopic coordinate set and a second stereoscopic coordinate set of the positioning mark according to the CT image containing the positioning mark and the mixed reality image containing the positioning mark, respectively; The matrix module is used to determine a coordinate transformation matrix according to the first three-dimensional coordinate set and the second three-dimensional coordinate set of the positioning mark; The conversion module is used to convert a first stereoscopic coordinate set of the surgical path into a second stereoscopic coordinate set of the surgical path through the coordinate conversion matrix; the first stereoscopic coordinate set of the surgical path is determined according to a stereoscopic image of the surgical path, and the stereoscopic image of the surgical path is determined according to a CT image containing a lesion area; The imaging module is used for performing imaging display according to the second stereoscopic coordinate set of the surgical path; The positioning mark includes: a first positioning mark and a second positioning mark; the relative position between the first positioning mark and the second positioning mark is fixed; the first positioning mark is used for identification and imaging by a CT machine, and the second positioning mark is used for identification and imaging by a mixed reality head-mounted device; The step of determining a coordinate transformation matrix according to the first three-dimensional coordinate set and the second three-dimensional coordinate set of the positioning mark comprises: Determine the first three-dimensional coordinate set of the plurality of second positioning marks according to the first three-dimensional coordinate set of the plurality of first positioning marks and the relative position data between the plurality of first positioning marks and the second positioning marks used in conjunction with them; the relative position data indicates the relative position between the first positioning mark and the second positioning mark used in conjunction with them; Based on the first stereoscopic coordinate set and the second stereoscopic coordinate set of the plurality of second positioning marks, a coordinate conversion matrix between the first stereoscopic coordinate system based on which the CT image is imaged and the second stereoscopic coordinate system based on which the mixed reality image is imaged is determined.

6. The device according to claim 5, characterized in that The image module is also used to image the positioning mark through a CT machine and a mixed reality head-mounted device to obtain a CT image containing the first positioning mark and a mixed reality image containing the second positioning mark.

7. The device according to claim 6, characterized in that Each of the positioning marks includes the same number of first positioning marks and second positioning marks, each of the first positioning marks is used in conjunction with one of the second positioning marks, and the relative positions of the first positioning marks and the second positioning marks used in conjunction are fixed.

8. An electronic device, characterized in that: The electronic device comprises a processor and a memory, wherein the memory is used to store a software program, and the processor runs the software program stored in the memory so that the electronic device implements the method according to any one of claims 1 to 4.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium is used to store program codes executed by a processor, wherein the program codes include instructions for implementing the method according to any one of claims 1 to 4.

10. A computer program product, characterized in that The computer program product comprises program codes, and when a computer runs the computer program product, the computer executes the method according to any one of claims 1 to 4.

Images