A surgical assistance system and method based on three-dimensional reconstruction of medical images

By designing a multi-module three-dimensional reconstruction system for medical imaging, collecting and updating surgical image data, the fatigue and operational errors of surgeons caused by surgical assistance systems in the existing technology are solved, and high-precision and high-efficiency surgical assistance is achieved, suitable for Various types of surgery.

CN114795465BActive Publication Date: 2024-11-22ZHEJIANG UNIV
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202210614322.X
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2022-05-31
Publication Date
2024-11-22
Estimated Expiration
2042-05-31

AI Technical Summary

Technical Problem

The existing surgical assistive technology based on three-dimensional reconstruction of medical imaging has problems such as easily causing fatigue of the surgeon, not suitable for large surgeries with long spans, poor picture clarity and risk of operational errors.

Method used

A system including control terminal, acquisition module, input module, generation module, logic module and output module was designed to create a three-dimensional model by collecting image data through multiple channels, and update the image data in real time during the operation, using thermal sensing scanning and The micro camera collects data, combines the neuron activation function to perform hierarchical stacked space calculation of image data, providing accurate three-dimensional image data and real-time surgical instrument position information.

Benefits of technology

It significantly reduces the risk of surgery, improves the accuracy and efficiency of surgery, is suitable for large organs, secondary surgery and first aid surgeons, reduces the fatigue of the surgeon, and assists more accurate surgery through real-time image updates.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN114795465B_ABST
    Figure CN114795465B_ABST
Patent Text Reader

Abstract

The present invention relates to the field of medical auxiliary technology, and in particular to a surgical auxiliary system and method based on three-dimensional reconstruction of medical images, comprising a control terminal, which is the main control end of the system and is used to issue execution commands; an acquisition module, which is used to acquire surgical wound image data; an input module, which is used to input the image data taken by all medical equipment during the surgical patient's visit to the system; a generation module, which is used to receive and identify the image data of the intercepted unit to generate a three-dimensional prototype model of the surgical patient's lesion surgical position; and a module for receiving the image data of each submodule of the acquisition module to synthesize an incomplete three-dimensional reality model. The present invention can acquire image data in multiple ways to establish a three-dimensional model and then output it. Both the accuracy of the three-dimensional image and the efficiency of image data replacement are better than those of the prior art, thereby greatly reducing the risk of surgery and bringing more convenience and efficiency to the surgeon.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of medical assistance technology, and in particular to a surgery assistance system and method based on three-dimensional reconstruction of medical images. Background Art

[0002] 3D reconstruction refers to the establishment of a mathematical model suitable for computer representation and processing of 3D objects. It is the basis for processing, operating and analyzing the properties of 3D objects in a computer environment. It is also a key technology for establishing virtual reality in computers to express the objective world.

[0003] With the development of science and technology, it is now also used in the medical field to assist doctors in performing surgery through the use of three-dimensional reconstruction of image models. This can reduce the vision requirements of the surgeon and focus more on the proficiency of the technique and surgical operation.

[0004] However, the current application of this technology has major flaws. Using AR equipment can easily cause more fatigue to the surgeon, limiting the duration of the operation. It is not suitable for large operations that take a longer time, and the picture clarity is poor, so there is a risk of operational errors for the surgeon during the operation. Summary of the invention

[0005] Technical issues solved

[0006] In view of the above-mentioned shortcomings of the prior art, the present invention provides a surgery assistance system and method based on three-dimensional reconstruction of medical images, which can effectively solve the problems raised in the above-mentioned background technology.

[0007] Technical Solution

[0008] To achieve the above objectives, the present invention is implemented through the following technical solutions:

[0009] In a first aspect, a surgical assistance system based on three-dimensional reconstruction of medical images comprises:

[0010] The control terminal is the main control terminal of the system and is used to issue execution commands;

[0011] An acquisition module is used to acquire surgical wound imaging data;

[0012] An input module for inputting image data captured by all medical devices during the patient's visit to the system;

[0013] A generation module is used to receive the image data of the recognition and interception unit to generate a three-dimensional prototype model of the surgical patient's lesion surgical position; and is used to receive the image data of the acquisition module submodule to synthesize an incomplete three-dimensional reality model;

[0014] The logic module is used to receive the 3D prototype model and the incomplete 3D reality model generated in the generation module, and match the image data size, viewing angle, and deflection angle of the two.

[0015] The output module is used to obtain the logic module matching structure, and to overlap and replace the incomplete 3D prototype model and the 3D reality model;

[0016] The output device is used to receive the three-dimensional model image data output by the output module for real-time image projection.

[0017] Furthermore, the acquisition module is provided with submodules, including:

[0018] Thermal induction scanning module, used to collect imaging data of the subcutaneous and adjacent tissue layers of the surgical wound;

[0019] The micro camera module is used to collect image data from all visible angles of the surgical wound.

[0020] Furthermore, the input module is provided with submodules, including:

[0021] The identification and interception unit is used to identify the image data captured by all medical devices during the consultation of all surgical patients input into the system in the input module; and to intercept the image data part of the lesion location in the corresponding image data of the surgical patients.

[0022] Furthermore, the output module is provided with submodules, including:

[0023] The storage unit is used to store the three-dimensional model image data output in real time during the current operation of the surgical patient.

[0024] Furthermore, the output module also includes the following submodules:

[0025] The comparison unit is used to compare the three-dimensional model image data stored in the storage unit, and use the three-dimensional model image data with the highest similarity as output content to provide it to the logic module through the output module as a data transmission path.

[0026] Furthermore, when the comparison unit is running, the input module and its submodules do not participate in the system operation, and the three-dimensional operation data generated by the comparison unit and the acquisition module provide the data conditions required for the operation of the generation module.

[0027] In a second aspect, a surgical assistance method based on three-dimensional reconstruction of medical images comprises the following steps:

[0028] Step 1: Obtain the image data of the surgical patients, and analyze the image data of the lesion location area collected by the patients;

[0029] Step 2: Determine the current state of the surgical patient, select the 3D model generation logic to establish and project the 3D model image data of the surgical patient in real time;

[0030] Step 3: Set up a convolutional layer data repository for surgical patients’ lesions and perform verification and pairing on the patients before surgery;

[0031] Step 4: Periodically update and reduce the data in the patient lesion data storage database;

[0032] Step 5: Install a built-in signal generator on the surgical instrument. During the operation, the surgical instrument used generates a three-dimensional model of the instrument through the built-in signal generator and synchronously inputs it into the three-dimensional model image data of the patient's lesion.

[0033] Furthermore, in the step 1, the analysis of the lesion location area image data of the operating data collected during the patient's visit covers one centimeter of the lesion and the lesion boundary.

[0034] Furthermore, after the verification and pairing of the convolutional layer data repository of the surgical patient lesions in Step 3 are performed, the process jumps to Step 4 for further execution regardless of whether there is a pairing result.

[0035] Furthermore, when the thermal induction scanning module collects the image data of the subcutaneous and adjacent tissue layers of the surgical wound, the image data layer stacking space is calculated using the following data:

[0036]

[0037] Where f represents the spatial activation function of the image data layer stacking;

[0038] θ represents the activation threshold of the neuron;

[0039] x represents the output value of each neuron that drives all its predecessor neurons;

[0040] w corresponds to the learnable parameters;

[0041] n represents the number of operations;

[0042] i represents the layer where the current computing image data is located;

[0043] y represents the image data layer stacking coordinates.

[0044] Beneficial Effects

[0045] Compared with the known public technology, the technical solution provided by the present invention has the following beneficial effects:

[0046] 1. The present invention provides a surgical assistance system based on three-dimensional reconstruction of medical images for use in medical surgery. The system can collect image data in multiple ways to establish a three-dimensional model and then output it. Both the accuracy of the three-dimensional image and the efficiency of image data replacement are better than the existing technology, thereby greatly reducing the risk of surgery and bringing more convenience and efficiency to the surgeon.

[0047] 2. The system of the present invention is suitable for large organ surgery and can well adapt to patients undergoing medical surgery, patients undergoing secondary surgery, and patients undergoing emergency surgery, and can provide accurate three-dimensional image data for all three.

[0048] 3. The present invention provides a surgical assistance method based on three-dimensional reconstruction of medical images for use in medical surgery. By adding a signal generator to the surgical instrument, the real-time update of the three-dimensional image data of the lesion location of the surgical patient can be assisted. The surgeon can observe the surgical instrument he operates in real time in this way, thereby further assisting the surgeon to perform the operation more accurately. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the drawings required for use in the embodiments or the prior art description are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention, and for ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0050] Figure 1 It is a structural schematic diagram of a surgical assistance system based on three-dimensional reconstruction of medical images;

[0051] Figure 2 It is a flowchart of a surgical assistance method based on three-dimensional reconstruction of medical images;

[0052] The numbers in the figure represent: 1. Control terminal; 2. Acquisition module; 21. Thermal sensing scanning module; 22. Micro camera module; 3. Input module; 31. Identification and interception unit; 4. Output module; 5. Connection module; 6. Output module; 61. Storage unit; 62. Comparison unit; 7. Output device. DETAILED DESCRIPTION

[0053] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings in the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0054] The present invention will be further described below in conjunction with the embodiments.

[0055] Example 1

[0056] A surgical assistance system based on three-dimensional reconstruction of medical images in this embodiment, referring to Figure 1 :include:

[0057] Control terminal 1 is the main control terminal of the system and is used to issue execution commands;

[0058] Acquisition module 2, used for acquiring surgical wound image data;

[0059] Input module 3, used to input the image data captured by all medical equipment during the patient's visit to the system;

[0060] Generating module 4 is used to receive the image data of recognition and interception unit 31 to generate a three-dimensional prototype model of the surgical patient's lesion surgical position; and is used to receive the image data of each submodule of acquisition module 2 to synthesize an incomplete three-dimensional reality model;

[0061] The logic module 5 is used to receive the three-dimensional prototype model and the incomplete three-dimensional reality model generated by the generation module 4, and match the image data size, viewing angle, and deflection angle of the two.

[0062] Output module 6 is used to obtain the matching structure of logic module 5, and perform model overlap and missing replacement for the incomplete three-dimensional prototype model and the three-dimensional reality model;

[0063] The output device 7 is used to receive the three-dimensional model image data output by the output module 6 for real-time image projection.

[0064] When used in this embodiment, the control terminal 1 controls the set module 2 to collect surgical wound image data, and then uses the input module 3 to input the system image data taken by all medical equipment during the surgical patient's visit. The generation module 4 then receives the image data of the identification and interception unit 31 to generate a three-dimensional prototype model of the surgical patient's lesion surgical position and synchronously receives the image data of the sub-modules of the acquisition module 2 to synthesize an incomplete three-dimensional reality model. Then the logic module 5 is post-operated to receive the three-dimensional prototype model and the incomplete three-dimensional reality model generated in the generation module 4, and match the image data size, viewing angle, and deflection angle of the two. Finally, the matching structure of the logic module 5 is obtained through the output module 6, and the three-dimensional prototype model and the incomplete three-dimensional reality model are overlapped and replaced. The output device 7 receives the three-dimensional model image data output by the output module 6 for real-time image projection for viewing by the leading doctor.

[0065] Example 2

[0066] In terms of specific implementation, based on Example 1, this example refers to Figure 1 The surgical assistance system for three-dimensional reconstruction of medical images in Example 1 is further specifically described as follows: the acquisition module 2 is provided with submodules, including:

[0067] Thermal induction scanning module 21, used to collect image data of the subcutaneous and adjacent tissue layers of the surgical wound;

[0068] The micro camera module 22 is used to collect image data of the surgical wound at all visible angles.

[0069] like Figure 1 As shown, the input module 3 is provided with submodules, including:

[0070] The identification and interception unit 31 is used to identify the image data captured by all medical devices during the consultation of all surgical patients input into the system in the input module 3; and to intercept the image data part of the lesion location in the corresponding image data of the surgical patients.

[0071] like Figure 1 As shown, the output module 6 is provided with submodules, including:

[0072] The storage unit 61 is used to store the three-dimensional model image data output in real time during the current operation of the surgical patient.

[0073] like Figure 1 As shown, the output module 6 also includes the following submodules:

[0074] The comparison unit 62 is used to compare the 3D model image data stored in the storage unit 61 , and use the 3D model image data with the highest similarity as output content to provide to the logic module 5 via the output module 6 as a data transmission path.

[0075] like Figure 1 As shown, when the comparison unit 62 is running, the input module 3 and its submodules do not participate in the system operation, and the three-dimensional operation data generated by the comparison unit 62 and the acquisition module 2 provide the data conditions required for the operation of the generation module 4.

[0076] Example 3

[0077] In terms of specific implementation, based on Example 1, this example refers to Figure 2 The surgical assistance system for three-dimensional reconstruction of medical images in Example 1 is further described in detail as shown below:

[0078] A surgical assistance method based on three-dimensional reconstruction of medical images comprises the following steps:

[0079] Step 1: Obtain the image data of the surgical patients, and analyze the image data of the lesion location area collected by the patients;

[0080] Step 2: Determine the current state of the surgical patient, select the 3D model generation logic to establish and project the 3D model image data of the surgical patient in real time;

[0081] Step 3: Set up a convolutional layer data repository for surgical patients’ lesions and perform verification and pairing on the patients before surgery;

[0082] Step 4: Periodically update and reduce the data in the patient lesion data storage database;

[0083] Step 5: Install a built-in signal generator on the surgical instrument. During the operation, the surgical instrument used generates a three-dimensional model of the instrument through the built-in signal generator and synchronously inputs it into the three-dimensional model image data of the patient's lesion.

[0084] like Figure 2 As shown, in step 1, the analysis range of the lesion location area image data of the operating data collected during the patient's visit covers one centimeter of the lesion and the lesion boundary.

[0085] like Figure 2 As shown, after the verification and pairing of the convolutional layer data repository of surgical patients' lesions in step 3 are performed, the process jumps to step 4 for further execution regardless of whether there is a pairing result.

[0086] When the thermal induction scanning module 21 collects the image data of the subcutaneous and adjacent tissue layers of the surgical wound, the following data is used to calculate the image data layer stacking space:

[0087]

[0088] Where f represents the spatial activation function of the image data layer stacking;

[0089] θ represents the activation threshold of the neuron;

[0090] x represents the output value of each neuron that drives all its predecessor neurons;

[0091] w corresponds to the learnable parameters;

[0092] n represents the number of operations;

[0093] i represents the layer where the current computing image data is located;

[0094] y represents the image data layer coordinates

[0095] In summary, the present invention provides a surgical assistance system based on three-dimensional reconstruction of medical images for use in medical surgery. The system can collect image data in multiple ways to establish a three-dimensional model and then output it. Both the accuracy of the three-dimensional image and the efficiency of image data replacement are better than those of the prior art, thereby greatly reducing the risk of surgery and bringing more convenience and efficiency to the surgeon. The system in the present invention is suitable for large organ surgery and can well adapt to patients undergoing surgery, patients undergoing secondary surgery, and patients undergoing emergency surgery, and can provide accurate three-dimensional image data for all three. The present invention also provides a surgical assistance method based on three-dimensional reconstruction of medical images for use in medical surgery. By adding a signal generator to the surgical instrument, the real-time update of the three-dimensional image data of the lesion position of the surgical patient can be assisted. In this way, the surgeon can observe the surgical instrument he operates in real time, thereby further assisting the surgeon to perform the surgery more accurately.

[0096] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. Such modifications or replacements will not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A surgical assistance system based on three-dimensional reconstruction of medical images, characterized in that: include: The control terminal (1) is the main control terminal of the system and is used to issue execution commands; An acquisition module (2) is used to acquire surgical wound image data; The acquisition module (2) is provided with submodules, including: A thermal induction scanning module (21) is used to collect image data of the subcutaneous layer and adjacent tissue layers of the surgical wound; A micro camera module (22) is used to collect image data of the surgical wound surface at all visible angles; An input module (3) is used to input image data captured by all medical equipment during the patient's visit to the system; The input module (3) is provided with submodules, including: The identification and interception unit (31) is used to identify the image data captured by all medical devices during the consultation of all surgical patients input into the system in the input module (3); and to intercept the image data portion of the lesion location in the image data corresponding to the surgical patients, The generation module (4) is used to receive the image data of the recognition and interception unit (31) to generate a three-dimensional prototype model of the surgical patient's lesion surgical location; and is used to receive the image data of the submodules of the acquisition module (2) to synthesize an incomplete three-dimensional reality model; The logic module (5) is used to receive the three-dimensional embryonic model and the incomplete three-dimensional real model generated by the generation module (4), and match the image data size, viewing angle, and deflection angle of the two. The output module (6) is used to obtain the matching structure of the logic module (5) and perform model overlap and missing replacement on the three-dimensional prototype model and the incomplete three-dimensional reality model; The output device (7) is used to receive the three-dimensional model image data output by the output module (6) for real-time image projection.

2. A surgical assistance system based on three-dimensional reconstruction of medical images according to claim 1, characterized in that: The output module (6) is provided with submodules, including: The storage unit (61) is used to store the three-dimensional model image data output in real time during the current operation of the surgical patient.

3. The surgical assistance system based on three-dimensional reconstruction of medical images according to claim 1, characterized in that: The output module (6) also Includes the following submodules: The comparison unit (62) is used to compare the three-dimensional model image data stored in the storage unit (61), and use the three-dimensional model image data with the highest similarity as output content to provide the data to the logic module (5) through the output module (6) as a data transmission path.

4. The surgical assistance system based on three-dimensional reconstruction of medical images according to claim 3, characterized in that: When the comparison unit (62) is running, the input module (3) and its submodules do not participate in the system operation, and the three-dimensional operation data generated by the comparison unit (62) and the acquisition module (2) provide the data conditions required for the operation of the generation module (4).

5. The surgical assistance system based on three-dimensional reconstruction of medical images according to claim 1, characterized in that: When the thermal induction scanning module (21) collects the image data of the subcutaneous layer and the adjacent tissue layer of the surgical wound, the image data layer stacking space is calculated by the following data: y=f( ); Where f represents the spatial activation function of the image data layer stacking; represents the activation threshold of the neuron; x represents the output value of each neuron that converts all its predecessor neurons; w corresponds to the learnable parameters; n represents the number of operations; i represents the layer where the current computing image data is located; y represents the image data layer stacking coordinates.

6. A surgical assistance method based on three-dimensional reconstruction of medical images, the method being an implementation method of the surgical assistance system for three-dimensional reconstruction of medical images as claimed in claim 1, characterized in that: The following steps are involved: Step 1: Obtain the imaging data collected from the patients during surgery and analyze the imaging data of the lesion location area collected from the patients during surgery; Step 2: Determine the current state of the surgical patient, select the 3D model generation logic to establish and project the 3D model image data of the surgical patient in real time; Step 3: Set up a convolutional layer data repository for surgical patients’ lesions and perform verification and pairing on the patients before surgery; Step 4: Periodically replace and reduce the data in the patient lesion convolutional layer data storage library; Step 5: Install a built-in signal generator on the surgical instrument. During the operation, the surgical instrument used generates a three-dimensional model of the instrument through the built-in signal generator and synchronously inputs it into the three-dimensional model image data of the patient's lesion.

7. The surgical assistance method based on three-dimensional reconstruction of medical images according to claim 6, characterized in that: In the step 1, the analysis of the image data of the lesion location area of ​​the image data collected during the patient's visit covers one centimeter of the lesion and the lesion boundary.

8. The surgical assistance method based on three-dimensional reconstruction of medical images according to claim 6, characterized in that: After the verification and pairing of the convolutional layer data storage library of the surgical patient lesions in Step 3 are performed, the process jumps to Step 4 for further execution regardless of whether there is a pairing result.